Feature-level Analysis of a Novel Smartphone Application for Smoking Cessation

Abstract

Background

Currently, there are over 400 smoking cessation smartphone apps available, downloaded an estimated 780,000 times per month. No prior studies have examined how individuals engage with specific features of cessation apps and whether use of these features is associated with quitting.

Objectives

Using data from a pilot trial of a novel smoking cessation app, we examined: (1) the ten most-used app features, and (2) prospective associations between feature usage and quitting.

Methods

Participants (n=76) were from the experimental arm of a randomized, controlled pilot trial of an app for smoking cessation called “SmartQuit,” which includes elements of both Acceptance and Commitment Therapy (ACT) and traditional cognitive behavioral therapy (CBT). Utilization data were automatically tracked during the 8-week treatment phase. Thirty-day point prevalence smoking abstinence was assessed at 60-day follow-up.

Results

The most-used features--quit plan, tracking, progress, and sharing--were mostly CBT. Only two of the ten most-used features were prospectively associated with quitting: viewing the quit plan (p=.03) and tracking practice of letting urges pass (p=.03). Tracking ACT skill practice was used by fewer participants (n=43) but was associated with cessation (p=.01).

Conclusions

In this exploratory analysis without control for multiple comparisons, viewing a quit plan (CBT) as well as tracking practice of letting urges pass (ACT) were both appealing to app users and associated with successful quitting. Aside from these features, there was little overlap between a feature's popularity and its prospective association with quitting. Tests of causal associations between feature usage and smoking cessation are now needed.

Cigarette smoking kills six million people worldwide each year (1), and effective interventions with the capacity to reach the ever-growing population of smokers are greatly needed. Mobile technology is poised to expand the current reach of cessation assistance. Smartphones are increasingly becoming the mobile technology platform of choice, owned by over half of the adults in the US (2) and one billion users worldwide (3), with new mobile phone purchases favoring smartphones over other mobile platforms by a margin of three to one (4). Smoking cessation assistance delivered through smartphone applications (“apps”)—software installed and run locally on the phone—can reach smokers who are not willing or able to access other modalities of treatment to help them quit. A key advantage of smartphone applications is their low cost and provision of on-the-spot coaching with engaging, multimedia content.

Currently, there are at least 400 smartphone apps for smoking cessation available (5, 6). Smoking cessation apps are downloaded an average of 780,000 times per month (5). Despite their popularity, no prior studies have examined use of specific features of smoking cessation apps and whether this use predicted quitting. This type of analysis is a critical first step toward identifying the active ingredients of smartphone apps for smoking cessation and understanding whether users are being adequately exposed to these active ingredients. Although this has not been tested with smartphone apps, prior research has demonstrated a discrepancy between the features of a technology-delivered intervention that are most popular with users and those that are predictive of outcome. For example, a recent study of the BecomeAnEX.org website showed that the site's cigarette tracker, quit plan, and tool for coping with smoking triggers were three of the most highly-accessed features on the site, yet none were significantly predictive of 30-day abstinence (7).

The current study addresses this serious gap in the literature by conducting a feature-level analysis of an innovative smartphone-delivered behavioral intervention for smoking cessation. SmartQuit is the first smoking cessation app to follow the principles of Acceptance and Commitment Therapy (ACT). Applied to smoking cessation, the focus of ACT is to increase the user's willingness to accept the physical, emotional, and mental states (e.g., physical urges, anxiety, or thoughts about smoking) that accompany smoking cessation while committing to engage in values-based behavior change (8, 9). Prior studies of ACT for smoking cessation have provided support for its efficacy relative to both pharmacotherapy alone (10) and behavioral interventions based on principles of traditional cognitive-behavioral therapy (CBT) (8, 11-13), which focus on developing skills to avoid or control smoking triggers rather than on developing skills to accept triggers.

In SmartQuit, ACT-specific exercises are grouped into three main categories of interventions designed to enhance motivation, provide skills for accepting urges to smoke, and cope with slips. SmartQuit also includes content that is not specific to ACT, but has some overlap with traditional CBT (14). These CBT features including self-monitoring with feedback (e.g., tracking smoking, viewing progress), goal-setting (e.g., creating a quit plan), positive reinforcement (e.g., earning badges for using the program and quitting smoking), and social support (e.g., sharing progress via email, text message, or social media).

The present study is a preliminary analysis using data from a randomized controlled pilot trial comparing SmartQuit with the National Cancer Institute's (NCI) QuitGuide app. Although the study was not powered to detect a statistically significant effect, the ACT SmartQuit app produced a promising quit rate compared with the NCI QuitGuide app (13% vs. 8%) (13). Due to certain technical features of the NCI's QuitGuide app, it was not feasible to obtain utilization data for participants assigned to the control intervention. Therefore, participants included in this post hoc analysis were only those who were randomized to receive SmartQuit. Given the current dearth of information on how users engage with apps for smoking cessation and how use of specific features predicts outcome, the focus of this exploratory study is to determine: (a) the most- used SmartQuit features, and (b) SmartQuit feature usage as a predictor of smoking abstinence at the 60-day follow-up.

Method

Participants

Criteria for participation in the randomized trial were: (a) age 18 or older; (b) smokes at least five cigarettes daily for the past 12 months, (c) wants to quit in the next 30 days; (d) interested in learning skills to quit smoking; (e) willing to be randomly assigned to either app; (f) resides in US; (g) knows how to login and download an app from Apple's “App Store”; (h) willing and able to read in English; (i) not using other smoking cessation interventions; and (j) has at least daily access to their own personal Apple iPhone 4, 4s, or 5. The iPhone was chosen as the only platform on which to test the app due to limited resources for programming. Participants not eligible for or interested in randomization were given a referral to treatment.

Because we were interested in determining how frequency of using specific features predicted outcomes, we excluded SmartQuit participants from the larger trial who never opened the app or didn't complete the follow-up survey (n=22). The average age of the included participants (n=76) was 41.8 years (SD=11.9). Approximately half (54%) were female, and the majority were Caucasian (87%), unmarried (67%), employed (58%), and had at least a high school education (91%). Twenty-four percent smoked one pack of cigarettes per day or more, and most (74%) had smoked for 10 years or more.

Procedures

All study procedures were approved by the Fred Hutchinson Cancer Research Center Institutional Review Board. Recruitment occurred between March 2013 and May 2013. Participants were recruited primarily through press releases, emails, advertisements on Facebook and Google, and a recruitment website developed for the study. We screened 738 potential participants. Of these, 400 were eligible, 340 provided informed consent, 205 completed a required phone confirmation, and 196 were randomized into the trial (N=98 to SmartQuit). Reasons for exclusion are detailed elsewhere (13). The proportions of exclusions from initial screening through randomization are very similar to our own (8) and another researcher's (15) web-delivered intervention trials.

Detailed app utilization data during the 8-week treatment period for SmartQuit were collected by 2Morrow Inc., the Seattle-based mobile software development company contracted to program the app. At 60-days post-randomization, participants were asked to complete a follow-up survey. Multimodal surveys were employed to maximize data completeness. Participants were first sent an email with a link to complete the follow-up survey online. An additional two reminders were sent by email. For those not responding to the email request, research staff contacted the participant to administer the survey by phone. If the phone contact was not successful, a paper copy of the survey was mailed along with a $2 bill pre-incentive to encourage participation. All participants who completed the follow-up survey were mailed a $25 incentive. Eighty of the 98 SmartQuit participants (82%) completed the 60-day follow-up survey. Of the 80 follow-up completers, 76 individuals opened the app at least once and are therefore included in the present analysis.

Outcomes

App utilization

Utilization data were log-in records collected from the secured server. Utilization metrics included the number of unique users who accessed each feature and the number of times the feature was used.

Smoking cessation

The primary smoking cessation outcome was 30-day point prevalence abstinence at the 60-day follow-up. Smoking abstinence was self-reported, as biochemical verification is not practical or necessary in population-based smoking cessation studies where there is no face-to-face contact with participants (16) and is not consistent with standard practice for population-based interventions (17, 18).

Description of the SmartQuit Application

The SmartQuit app includes features to plan, track, and visualize behavior change as well as to learn new strategies for accepting the difficult thoughts, feelings, and physical sensations (e.g., cravings) that can arise while trying to quit smoking. The Quit Plan feature (see screenshot in Figure 1) helps users to select a quit date, identify values-based inspiration for quitting (e.g., for family-related values, uploading a picture of one's children), develop specific behavioral strategies for reducing smoking, and consider use of pharmacotherapies to support cessation. Users are prompted to create this quit plan the first time they open the app. The three-category main menu of quitting strategies (see screenshot in Figure 1) includes Staying Motivated, Having an Urge, and I Slipped. Each of these menus contains ACT-specific exercises that are designed to enhance commitment to quitting by exploring and evoking values (Staying Motivated), enhancing willingness to feel an urge to smoke and let it pass (Having an Urge), and building self-compassion as an adaptive response to the experience of slipping up and smoking during a quit attempt (I Slipped). Exercises are presented as short video or audio, and all are accompanied by a text transcript.

Figure 1.

Screenshots of the app's: (A) main menu page, (B) quit plan feature, and (C) tracking feature.

At the bottom of the main menu are the following selections of features: The Tracking feature has users record number of cigarettes smoked, use of cessation medications, and use of two ACT-specific skills: number of times that they let urges pass (i.e., having an urge but not acting on it by smoking) and number of times they practiced one of the program's ACT tips for quitting (e.g., strategies for staying motivated, having an urge, or handling slips). The Progress feature provides visuals of tracked behaviors over time in calendar, chart, and location (i.e., from GPS tagging) form as well as a list of the progress badges earned by the user (e.g., for completing a quit plan, letting an urge pass, or using the app a specified number of times). The Sharing feature allows users to report their progress toward quitting through email, Twitter, Facebook, or text message. The Notes feature allows the user to create a date- and time-stamped journal entry.

To increase utilization, the app included twice-daily reminders that appeared on the phone as push notifications. The morning reminder suggested that the user try one of the ACT exercises, which was randomly selected without replacement from the list ACT exercises under the Staying Motivated, Having an Urge, and I Slipped menus. The evening reminder prompted the user to complete tracking for that day (i.e., tracking of smoking, medication use, practice of letting urges pass, and practicing ACT skills). Reminders were on by default but could be turned off by the user on the main screen. Study participants also received a weekly email during the 8-week treatment period reminding them to log in to their assigned app. Although participants were compensated for completing study outcome surveys, there were no incentives for use of the app. For further information about the app, including a depiction of its structure, see the main outcome report for the randomized, controlled trial of SmartQuit vs. QuitGuide (13).

Statistical Analyses

As an indicator of feature usage, the SmartQuit features were ranked according to the number of unique participants who accessed them. Due to the sizable number of app features (n=41), only the top ten most-used features are reported in the manuscript. As a secondary indication of feature usage, we also report the median (due to a skewed distribution) number of uses per participant for the most-used features. Logistic regression models were used to predict 30-day point prevalence smoking abstinence at the 60-day outcome assessment, with frequency of use of each app feature as a binary predictor (due to the skewed distribution, and to aid interpretation of findings). Thus, the binary predictor was high vs. low use, based on a median split of number of uses of each feature. To address potential confounding of the relationship between feature use and smoking cessation outcome, we conducted sensitivity analyses in which we controlled for any baseline demographic or clinical characteristic that was associated with the use of the specific feature as well as with cessation outcome. Potential covariates included age, gender, education (high school or less versus post-secondary), and heavy smoking (i.e., one pack or more per day). Due to the exploratory, hypothesis-generating focus of the study, we used a p-value of .05 for statistical significance, with no adjustment for multiple tests (19). All analyses were conducted using R, version 3.0.1 (20).

Results

What SmartQuit app features were most used?

Table 1 shows the frequency of use of the top ten app features, ranked according to the number of participants who accessed the feature at least once. The top eight ranked features were the only ones out of more than 40 total app features for which the median number of uses per participant was greater than one. Thus, these features were not only accessed by the highest number of users, they also tended to be accessed repeatedly by users. Notably, eight out of the top ten most popular features were based on traditional CBT: developing a quit plan (#1), tracking (tracking smoking, #2; viewing locations of tracking, #7); viewing progress (in calendar form, #3; in chart form, #5; in the form of badges earned, #9), sharing (#4), and journaling (opening notes, #10). The only two ACT-specific features represented in the top ten list were tracking practice of letting an urge pass (#6) and viewing an ACT exercise for staying motivated (#8).

Table 1.

Top ten SmartQuit app features, by number of unique users

Feature	ACT-specific?	n a	Median no. of uses	OR (95% CI)b
1. Viewed quit plan overview	No	76	8	11.1 (1.3-94.2)*
2. Tracked smoking	No	67	24	0.1 (0.1-0.9)*
3. Viewed progress in calendar	No	65	4	2.8 (0.6-12.1)
4. Viewed sharing page	No	64	2	2.5 (0.6-10.7)
5. Viewed progress in chart	No	64	6	1.7 (0.4-7.1)
6. Tracked practice of letting urges pass	Yes	61	9	10.5 (1.2-88.6)*
7. Viewed location of tracking	No	58	2	2.0 (0.5-8.3)
8. Staying motivated exercise	Yes	56	2	3.4 (0.8-14.6)
9. Viewed badges earned	No	55	1	1.5 (0.4-6.3)
10. Opened notes	No	51	1	1.0 (0.2-4.3)

Note: ACT=Acceptance and Commitment Therapy.

^aNumber of participants who opened the feature at least once

^bOdds ratios, with 95% confidence intervals, from models predicting 30-day point prevalence smoking abstinence at the 60-day follow-up from high vs. low use of each feature.

^*p < .05

What SmartQuit app features were most predictive of cessation?

App features that demonstrated the strongest prospective prediction of smoking abstinence are shown in Table 2. Two of the features were ACT-specific tracking features—i.e., tracking ACT skills practice (p=.01) and practice of letting urges pass (p=.03)—whereas viewing the quit plan (p=.03) was a traditional CBT feature. Only one baseline variable was associated with use of one of these features as well as well as smoking abstinence—male gender predicted greater use of the tracking ACT skills practice exercise (p=.02) as well as abstinence (p=.02). Controlling for gender did not change conclusions about the relationship between tracking ACT skills practice and smoking abstinence (p=.009, OR=18.8, 95% CI=2.1-171.3).

Table 2.

SmartQuit app features that prospectively predicted smoking abstinence (p < .05)

Feature	ACT-specific?	p	OR (95% CI)b	n a	Median no. of uses
1. Tracked ACT skill practice	Yes	.01	16.4 (1.9-139.1)	43	1
2. Viewed Quit Plan overview	No	.03	11.1 (1.3-94.2)	76b	8
3. Tracked practice of letting urges pass	Yes	.03	10.5 (1.2-88.6)	61b	9

Note: ACT=Acceptance and Commitment Therapy.

^aNumber of participants who opened the feature at least once.

^bFeature ranked among top ten in number of unique users.

Several additional ACT features were predictive of smoking abstinence at trend level: (1) viewing a Staying Motivated video (p=.06, OR=4.1, 95% CI=0.9-17.6, n=15 users), in which a former smoker describes how work-related values contributed to his motivation to quit; (2) viewing the Having an Urge exercise entitled Leaves on a Stream (p=.06, OR=4.1, 95% CI=0.9-17.6, n=15 users), which teaches participants to take a step back from their thoughts by nonjudgmentally observing their flow as if they were leaves on a stream, and, (3) viewing the Having an Urge exercise entitled Chinese Handcuffs (p=.07, OR=4.3, 95% CI=0.9-21.1, n=10 users), which metaphorically illustrates the problem of struggling against, rather than allowing, urges to smoke.

Finally, it should be noted that, in addition to the features listed in Table 2, tracking smoking was also associated with cessation, but in the opposite direction. That is, tracking smoking predicted a lower likelihood of quitting at 60-day follow-up (data not shown in table; p=.04, OR=0.11, 95% CI= 0.1-0.9). The probable cause of this unexpected finding is discussed below.

Discussion

This study explored user engagement with features of a novel smartphone application for smoking cessation and evaluated whether use of these features predicted successful quitting. Results indicated that the traditional CBT features—particularly the quit plan, tracking progress, viewing progress, and sharing—tended to be the most frequently used features. Only a minority of the ten most popular app features predicted successful cessation (i.e., CBT feature of viewing the quit plan and ACT feature of tracking practice of letting an urge pass). Tracking ACT skills practice was also prospectively predictive of cessation, although this feature was used by less than half of the participants (n=43). Surprisingly, we found that one of the more popular features, tracking smoking, predicted a lower likelihood of quitting. However, the most likely interpretation of this finding is that it is an artifact of the method of recording smoking behavior within the app (i.e., requiring active tracking only when the participant has smoked), confounding the use of the feature with the outcome of smoking.

Taken together with the findings of a similar study focusing on a web-based intervention for smoking cessation (7), we conclude that features that are popular with users are not necessarily predictive of quitting. These findings further highlight the need for rigorous evaluation of the effectiveness of smoking cessation apps, as it is possible that (a) the most popular apps have features that are appealing to users yet ineffective for smoking cessation, and (b) even those app features that follow evidence-based principles for in-person interventions (e.g., getting social support) may not be effective when translated into a mobile health intervention (e.g., getting social support by sharing via text, email, or social media). This latter point is important, as recent reviews of the quality of smoking cessation apps (5, 6) have evaluated them exclusively based on their adherence to the US Public Health Service Clinical Practice Guidelines (21), which were developed to guide in-person interventions and thus are not ideal for assessing app quality. There is now a substantial need for research to determine how this highly-used mobile technology can provide effective treatment to the sizable population of smokers who are now turning to their smartphones for assistance with cessation (i.e., 780,000 per month; (5)).

Results of this study should be interpreted in light of key limitations. First, as noted previously, the finding that tracking smoking predicted a worse smoking cessation outcome should be interpreted with caution because use of this feature was confounded with the outcome of smoking. Second, associations between feature usage and outcome are correlational. Additional studies are needed to experimentally manipulate the availability of app features that predicted quitting in order to determine if they are causally linked to cessation. Third, the structure and appearance of a smartphone app, or of any other technology-delivered intervention, can affect user behavior. Thus, results of this study should be interpreted in light of the possibility that feature usage may have also been influenced by factors other than interest in specific content (e.g., placement within the program, aesthetics, etc.). Fourth, our findings regarding an association between feature usage and smoking cessation require replication because: (1) we did not correct for multiple comparisons, inflating the risk of type I error, (2) the small sample size and modest overall quit rate for SmartQuit (13%) reduces precision of the effect size estimates, and (3) there is a possibility that the observed associations could be driven by a third variable that we didn't measure, such as motivation to quit or self-efficacy. Finally, our outcome data included only self-reported smoking abstinence measures, and we were consequently unable to determine whether feature usage facilitated progress toward quitting during the treatment period (e.g., reducing the number of cigarettes smoked per day) or the extent to which self-reported abstinence was consistent with biochemically verified abstinence.

Continued efforts to understand user engagement with smartphone apps for smoking cessation and how use of specific app features relates to quitting are critical for the design of effective interventions on this new platform. Supplementing objective measures of utilization (i.e., server-recorded opening of app features) with self-report measures will be a critical step toward gaining a more complete understanding of how and why users preferentially use specific features of an app. Nonetheless, results of the present study provide the first evidence that: (a) there is little overlap between an app feature's popularity and its association with quitting, and (b) viewing a quit plan and tracking practice of letting urges pass are both appealing to app users and predictive of quitting. Given that these findings are preliminary and hypothesis-generating, we believe that an important next step toward developing effective smartphone apps for smoking cessation is to test causal hypotheses regarding the association between specific features and cessation. Identification of these causal relationships would provide two critical scientific advances: (1) a method for evaluating the quality of the hundreds of existing apps and (2) an empirical foundation for developing new apps with features that are both engaging and efficacious.