Effectiveness of Financial Incentives for Longer-Term Smoking Cessation: Evidence of Absence or Absence of Evidence?

Conventional wisdom is that financial incentives are highly effective in changing short-term behavior but are much less effective in changing longer-term behavior. A Cochrane collaboration review of financial incentives for smoking cessation in workplace settings concluded in 2008 that “Incentives and competitions do not appear to enhance long-term cessation rates. Early success tended to dissipate when the rewards were no longer offered, and the normal relapse pattern re-established itself.”¹ In this manuscript, we examine the evidence on effectiveness of financial incentives for smoking cessation in achieving long-term smoking cessation in workplace settings. Smoking is an example of a behavior where long-term behavior change is of obvious import, and we will examine existing reviews to determine whether there is evidence of ineffectiveness or simply lack of evidence of effectiveness.

We undertook a review of the studies included in the meta-analysis conducted as part of the Cochrane review¹ of randomized trials of financial incentives for smoking cessation. We assessed the nine trials reported in the Cochrane review, along with two of our own studies. For each study, we reviewed the magnitude of the incentives offered and the outcomes, focusing on quit rates in the incentive and control groups at 6 and 12 months following randomization (or the assessments nearest those intervals). For cluster-randomized studies, we calculated an “effective N” based on the reported intraclass correlation; we used the actual sample size for non–cluster-randomized studies. We tabulated the quit rates for the control group at the follow-up points closest to 6 and 12 months and calculated the incentive group quit rate (if not provided directly) using the reported odds ratios (ORs). Note that many of these reported ORs derived from adjusted models (e.g., for demographic or other covariates) and we used these adjusted ratios whenever available. Finally, we calculated the detectable OR comparing the quit rates in the incentive and control groups. We used a simple comparison of binomial proportions to determine the detectable increase in quit rate in the incentive group assuming the observed baseline control group rate, the effective sample size, a two-sided .05-level test, and 80% power, and then calculated the detectable OR from these proportions.

Table 1 presents the studies assessed, with their actual and effective sample sizes, control and incentive (adjusted) quit rates, observed adjusted ORs, and detectable ORs with 80% power. The table also provides average attrition rates across arms, and an indication of whether any attempt was made to accommodate missing values in the analysis. The studies included in the Cochrane review were generally not designed with sample sizes large enough to detect clinically significant differences in long-term quit rates (see Table 1). Those that randomized by individual had extremely small sample sizes (60 in Paxton,⁶ 47 in Rand et al.,⁷ 120 in Gallagher et al.,³ 175 in Shoptaw et al.⁸) and those that used cluster randomization had relatively small sample sizes once the intraclass correlation was accounted for.⁵ As illustrated (Table 1), most of the previous studies had effective Ns less than 190, leaving them with enough power to detect only fairly large differences in smoking cessation rates. The detectable OR with 80% power using the effective sample size was greater than 3.0 in most cases, with detectable ORs ranging from 2.6 to 20.0 for studies other than our 2009 study. This means that these studies could generally not detect a doubling or even a tripling of smoking cessation rates. Note that the differences in detectable ORs between the 6- and 12-month outcomes in the same study are a result of different control group quit rates at those assessments. It is important to emphasize that given the very low rates of smoking cessation typically achieved (5%–15% in Table 1), even small relative increases in success rates would have enormous public health significance; ideally studies would be able to detect 10%–20% increases in cessation rates, e.g., a detectable relative risk of 1.1–1.2.

Table 1.

Effective Sample Sizes and Detectable Odds Ratios (ORs) of Financial Incentive and Smoking Cessation Workplace Studies—6- and 12–Month Outcomes

Study	N	Effective N*	Average Attrition Rate	Control Quit Rate	Adjusted Incentive Quit Rate	Observed OR adj*	Detectable Incentive Quit Rate	Detectable OR
6-month outcomes
Jason et al.2†	561	517	0.45‡	0.075	0.174	2.59	0.157	3.01
Gallagher et al.3	120	120	0.48‡	0.050	0.067	1.36	0.247	6.23
Gomel et al.4†	60	59	0.16	0.130	0.038	0.25	0.491	6.46
Klesges et al.5†	127	108	0.04	0.115	0.125	1.10	0.361	4.35
Paxton6	60	60	0.15	0.450	0.435	0.94	0.824	5.72
Rand et al.7	47	47	0.24‡	0.033	0.083	2.64	0.406	20.0
Shoptaw et al.8	85	85	0.20‡	0.100	0.025	0.23	0.380	5.52
Shoptaw et al.8E	89	89	0.20‡	0.100	0.168	1.82	0.374	5.38
Volpp et al.9	179	179	0.31‡	0.046	0.065	1.45	0.192	4.92
Volpp et al.10	878	878	0.14‡	0.118	0.209	1.98	0.188	1.73
Windsor et al.11	190	190	0.10‡	0.070	0.191	3.14	0.224	3.83
Windsor et al.11E	188	188	0.10‡	0.200	0.242	1.28	0.396	2.62
12-month outcomes
Jason et al.2†	561	517	0.50‡	0.157	0.203	1.37	0.261	1.90
Glasgow et al.12†	770	535	0.21	0.116	0.104	0.88	0.209	2.01
Gomel et al.4†	60	59	0.19	0.100	0.015	0.14	0.454	7.48
Shoptaw et al.8	71	71	0.20‡	0.111	0.054	0.46	0.430	6.04
Shoptaw et al.8E	71	71	0.20‡	0.061	0.027	0.43	0.355	8.47
Volpp10	878	878	0.26‡	0.050	0.147	3.28	0.103	2.18
Windsor 1988	190	190	0.10‡	0.060	0.171	3.23	0.209	4.14
Windsor B 1988	188	188	0.10‡	0.180	0.267	1.66	0.373	2.71

^*Calculated from Cahill and Perera.¹

^†Cluster-randomized trial.

^‡Some adjustment made for missing values.

^EThese studies used factorial designs; the two row represent the incentive effect at each level of the second factor.

All of the studies suffered from some subject attrition over time, ranging from about 5% to nearly 50%. Most studies made some attempt to adjust for this, by using regression-based adjustment approaches (Jason et al.,² Shoptaw et al.⁸), by implementing an imputation procedure (assuming that subjects lost to follow-up failed to achieve cessation; Volpp et al.,^9,10 Windsor et al.¹¹), or by using last observation carried forward (Gallagher et al.³).

In addition, the magnitude of the incentives for smoking cessation used in previous studies was generally too small to constitute an adequate test of incentives, as many of the studies used lotteries with small expected value of payouts (some of them about $10 total; Table 2). The largest previous study^2,5 used incentives of $175, worth about $400 in 2007 dollars, and did find a significantly higher quit rate at 6 months (OR 2.59, 95% confidence interval 1.29, 5.21), though it had lower power than our study and used smaller incentives.

Table 2.

Incentive Designs for Studies With 12-Month Outcomes

Study	Inflation Adjusted Incentive Amount (Original)	Incentive Design	Approach to Detection
Jason et al.	$253 ($175)	$1/d for each day of abstinence.	CO test
Gallagher et al.3	$713 ($480)	$20 for each negative CO test at weekly visits (1–4), $40 every 2 wk at weeks 6–12, $60 every month at weeks 16–24, $80 at final visit at week 36.	CO tests
Glasgow et al.12	Maximum approximately $181 ($122)	$10 each CO reading <9 ppm at monthly meetings × 1 y. Lottery prize for 1 smoker worth $5 to $50. 3 prize drawings ($200, $100, $50) per worksite (average 329 employees per worksite).	CO and cotinine tests
Gomel et al.4	Maximum approximatel y $178 ($120)	Two lottery draws for a $40 voucher in 10 wk (EV = $0.20); $40 for meeting 3-mo goals; $1,000 for station with largest percentage of participants (each station has at least 12 employees).	Plasma cotinine
Klesges et al.5	$57 ($30)	Combination of team and individual incentives based on smoking cessation at weekly intervals and then at 6 mo.	CO /saliva thiocyanate
Paxton6	Approximately £118 (£40)	Deposit contract with £20 repaid at £5 /wk of following 4 wk if no smoking (month 1). Month 2: £20 repaid at £10 every 2 wk if no smoking.	Urine cotinine analysis
Rand et al.7	$396 ($229)	$25 for completion of abstinence week. Breath samples 2×/wk—$4 for each CO value <11 ppm × 26 wk.	CO
Shoptaw et al.8	Maximum $447.50 ($648)	Vouchers worth $2 for initial CO level ≤8 ppm, increased by $0.50 for consecutive samples, with bonus of $5 each third consecutive negative sample. If relapsed, next negative sample worth $2. Returned to point prior to positive sample following 3 consecutive negative samples.	CO breath tests
Volpp et al.9	$213 ($200)	$20 for attendance at each of 5 smoking cessation classes (maximum $100), $100 for smoking cessation 30 days after program completion.	Urine cotinine tests
Volpp et al.10	$750	$100 for completion of smoking cessation program, $250 for cessation within first 6 mo, $400 for cessation for additional 6 mo.	Saliva cotinine tests; urine cotinine in cases where patients on NRT
Windsor et al.11	$91 ($50)	$25 following 6 wk of cessation, additional $25 following 6 mo of cessation.	Saliva thiocyanate

In 2009, we completed and published a study that involved nearly 900 employees of General Electric.¹⁰ In this study, we offered financial incentives worth a total of $750 if study participants completed a smoking cessation program and reported prolonged abstinence by the 9–12-month point postenrollment. In this study, we had sufficient power to detect a relative risk of 1.59, considerably smaller than most of the studies listed above. Statistically significant relative risks were detected at both 6 and 12 months. Our study is the only one to date that found larger observed relative rates of quitting in the intervention group at 12 months than what was detectable based on the available sample size. There are a variety of possible explanations for the discrepancy between our study and those of others. The first is simply that our sample size afforded greater statistical power than nearly all of the other studies, so if a difference existed we were more likely to be able to measure it. The second is that the incentives we offered were larger than those offered in nearly all of the other studies. A third may be that the employer climate between 2004 and 2008 was more hospitable to a smoking cessation and financial incentive intervention than in many of the earlier studies.

In summary, the existing evidence on financial incentives and smoking cessation is quite limited, and the preponderance of negative studies is potentially quite misleading. None of the studies had sufficient statistical power to detect differences in smoking cessation rates anywhere near the minimum threshold of clinical significance, as most could not detect even a doubling or tripling of rates. Summaries of the literature really should highlight that the effectiveness of incentives for longer-term cessation remains a largely open question. The studies that have been conducted to date provide a good illustration of the common maxim, “Absence of evidence is not evidence of absence.” Rather than proving a lack of effectiveness, the studies to date simply have been inadequately powered to address the question of whether incentives increase long-term smoking cessation rates.

A second major issue is the magnitude of the incentives used in previous studies. To adjust for changes in the purchasing power of a dollar over time, monetary amounts were adjusted for the growth in the government-provided U.S. consumer price index between the year in which the study was originally published and the publication of the Volpp et al.¹⁰ study in 2009. Nonetheless, it is apparent that many used small incentives (Table 2), and negative studies in this context simply indicate that weak incentives are ineffective at changing behavior. Furthermore, some interventions have been designed to achieve higher rates of smoking cessation in the short term but have not structured their incentive programs towards achieving higher long-term quit rates.⁹

This study is limited by the fact that we do not really know whether incentives work for longer-term behavior change because there are so few adequately powered studies that examine this issue. Furthermore, it is entirely possible that the likelihood of longer-term behavior change varies systematically by behavior, as socioenvironmental factors may make longer-term behavior change more difficult for obesity, for example, than for smoking.

The relationships between the size and structure of incentive payments and smoking cessation rates remain important empirical questions to be addressed in future research. It is possible that larger — or smaller — payments could be more cost-effective to employers in improving smoking cessation rates, but the optimal design is very much an open question, as extension of the incentives beyond 12 months may produce higher cessation rates. Other areas where further research is needed include (1) examination of the relative effectiveness of tying rewards to insurance premium adjustments vs. separate rewards, (2) the relative effectiveness of rewards (“carrots”) vs. penalties (“sticks”), (3) the effectiveness of lottery rewards vs. direct payments for smoking cessation, and (4) the effectiveness of monetary vs. nonmonetary rewards or social recognition vs. or in conjunction with pharmacological therapies.

This study highlights that in the context of smoking cessation we have much to learn about the effectiveness of incentives; what is clear is that the current literature provides inadequate evidence to make a determination. More adequately powered studies that test variations in incentive design, magnitude, and payment schedule are needed to determine the effectiveness of this approach in increasing the rate of smoking cessation.