Abstract
Introduction
Studies of the users’ engagement with smoking cessation application (apps) can help understand how these apps are used by smokers, in order to improve their reach and efficacy.
Objective
The present study aimed at identifying the best predictors of the users’ level of engagement with a smartphone app for smoking cessation and at examining the relationships between predictors and outcomes related to the users’ level of engagement with the app.
Methods
A secondary analysis of data from a randomized trial testing the efficacy of the Stop-Tabac smartphone app was used. The experimental group used the “full” app and the control group used a “dressed down” app. The study included a baseline and 1-month and 6-month follow-up questionnaires. A total of 5,293 participants answered at least the baseline questionnaires; however, in the current study, only the 1,861 participants who answered at least the baseline and the 1-month follow-up questionnaire were included. Predictors were measured at baseline and after 1 month and outcomes after 6 months. Data were analyzed using machine learning algorithms.
Results
The best predictors of the outcomes were, in decreasing order of importance, intention to stop smoking, dependence level, perceived helpfulness of the app, having quit smoking after 1 month, self-reported usage of the app after 1 month, belonging to the experimental group (vs. control group), age, and years of smoking. Most of these predictors were also significantly associated with the participants’ level of engagement with the app.
Conclusions
This information can be used to further target the app to specific groups of users, to develop strategies to enroll more smokers, and to better adapt the app’s content to the users’ needs.
Keywords: Smoking cessation apps, Tobacco dependence, Smartphone, Randomized controlled trial
Introduction
Tobacco smoking is one of the most important causes of mortality globally [1]. Smartphone applications (apps) for smoking cessation are expensive to develop, but they can reach thousands of smokers at a low cost per participant [2, 3]. In two separate recent meta-analyses of randomized trials, only one smoking cessation app (the same one in both studies) increased smoking cessation rates, and the overall conclusions of both meta-analyses, based on limited evidence, were in favor of no effect [2, 4]. Possible conclusions are that either the content or quality of the included apps was insufficient, or that the apps were of good quality but smokers did not use them frequently enough or during a long enough time, or they did not follow the apps’ advice and recommendations, or that even intensive use of the best possible app and optimal adherence to recommendations are insufficient to treat tobacco dependence [2, 4–6].
The quality and content of smoking cessation apps vary widely, and few of them have been scientifically tested [3, 7]. Problems that are common to many smoking cessation apps are low enrolment rates of the target audiences, high attrition rates, and low frequency of use among those enrolled [8–12], and low completion rates of tasks and modules [13]. This seriously limits the potential impact of those apps at population level [14].
Studies of user engagement with smoking cessation apps can provide useful information to develop more attractive apps or improve existing apps. This can ensure that apps are effective and used repeatedly by a large proportion of the target audience, and that they meet users’ needs.
While this topic has seldom been explored, the available studies show that engagement with smoking cessation apps is associated with user’s characteristics such as literacy skills, younger age, being a woman, not being depressed, lower levels of tobacco dependence, greater acceptance of craving, and perceived utility of the app [8–10, 14]. Identifying the characteristics of participants who are less likely to engage with the app can help develop elements targeted at these specific subgroups and improve their experience. For instance, apps can include gender- or age-specific content, video and audio files for those with low reading skills, and content targeted at those with high levels of cigarette dependence or low acceptance of craving.
Study Purpose
Therefore, the present study aims at identifying the best predictors of the users’ engagement with a smartphone application for smoking cessation and at exploring the associations between these predictors and utilization variables that were collected automatically by the app.
Research Questions
This was a secondary analysis of data from a randomized controlled trial testing the efficacy of the Stop-Tabac smartphone app for smoking cessation [6]. We addressed six questions: what are the most important predictors (in ranking order) of the number of different days when participants accessed the app (RQ1), of the number of times the app was opened (RQ3), and of the app use duration (i.e., interval in days between the first and last day the app was opened) (RQ5)? We also examined the relationships (association probabilities) between each predictor and each of the same three outcomes (RQ2, RQ4, and RQ6). Because this was an exploratory study, no hypotheses associated with the research questions were made, but all the tested associations made sense.
Methods
The Stop-Tabac App
The full Stop-Tabac app is based on the self-determination theory of behavior change [15], allowing high emphasis with autonomy (enhancing patient determination of behavior change, personalization of the app and rewards), competence (e.g., tools to enhance behavior change capabilities, tailored messages), and relatedness (e.g., with the app messages and with the app community, sharing progress). It is also built, in coherence with the Capability-Opportunity-Motivation-Behaviour (COMB) theory [16], trying as much as possible to reinforce capability and motivation. Like in some other digital tools [17], the app is adapted to include automated versions of several components offered by smoking cessation interventions such as assessment and feedback, motivational enhancement interventions, and relapse prevention strategies [18] as well as goal setting, action planning, and feedback in relation to goal achievement [19]. For instance, the Stop-Tabac app includes information pages, calculators (number of cigarettes not smoked, money and days of life gained since quitting), individually tailored (automatized) counseling reports, a quiz, phone numbers of quitlines, and a module on nicotine replacement therapy and e-cigarettes. The Stop-Tabac app is available at no charge in the Apple App Store and Google Play Store. It is a stand-alone intervention, and there was no human involvement or support in this intervention. It was rated among the best smoking cessation apps globally by two distinct studies independent from the authors [3, 20]. The control version of the app includes a few features that are liked by users (brief information pages, calculators of unsmoked cigarettes, money saved, and life expectancy gained since quitting). For more information, see description of the protocol trial [21].
Participants’ Characteristics
We enrolled 5,293 daily smokers in the randomized trial. However, in the present study we included only the 1,861 participants (35% of 5,293) who completed at least the baseline and 1-month follow-up questionnaire (treatment group n = 902, control group n = 959), because we used the information collected after 1 month to predict outcomes after 6 months. The 1,861 participants were included in all data analysis models built in the present study.
Recruitment and Sampling Procedures
The recruitment procedure was previously described [6, 20]. Briefly, eligible participants were adult daily cigarette smokers who lived in Switzerland or in France, and who set a quit date within 1 month of enrollment. Participants were enrolled via advertisements on the Internet. After downloading the app from the app stores, participants clicked on a link in the app leading to the online informed consent form and online screening and baseline questionnaire. Then, eligible participants received a code that enabled them to unlock either a full version of the Stop-Tabac smoking cessation app or a control version. Eligibility assessment was done automatically.
Data Collection
Participants answered follow-up questionnaires online 1 month and 6 months after their target quit date. Non-respondents received three reminders by e-mail, and then one text or WhatsApp message; then, they received the questionnaire by postal mail and the remaining non-respondents were contacted by phone. The questions were developed for the purpose of this study and were not submitted to an evaluation of their psychometric properties or to prior validation tests. The registration and consent form and the baseline and follow-up questionnaires are available here: https://archive.org/details/@stopdependance_ch.
Predictor Variables
Variables collected at baseline were country of residence, age, sex; experiment group category (full Stop-Tabac app or control version); smoking status, number of years as a smoker, number of cigarettes smoked per day, number of minutes between waking up and smoking the first cigarette of the day (which is an indicator of dependence), smoking other products, use of heated tobacco, electronic cigarettes and nicotine replacement medications (patch, gum, tablet, etc.), each on 4-point response scales, and a 2-item depression screening test (yes, no) [22]. The 1-month questionnaire covered the same variables as at baseline plus: usage of any smoking cessation app after entry in the study (yes, no), intention to stop smoking (3-point response scale), and perceived helpfulness of the app (6-point response scale).
Outcome Variables
The following information was collected automatically by the app itself in 100% of participants at the end of the study: (a) the number of different days when each participant used the app, (b) the number of times the app was opened, and the (c) the duration of app use (i.e., the interval in days between the first and last day the app was opened). Online supplementary material (for all online suppl. material, see www.karger.com/doi/10.1159/000530111) presents a table with descriptive statistics of the participants’ responses on the predictors and outcome variables.
Ethics
The study protocol was submitted to the Cantonal Ethics Committee in Geneva (number: Req-2018-00356) who answered that the app being no medical device, the study did not need their approval. The commission did therefore not review the protocol but wrote in an e-mail dated May 16, 2018, that “everything indicates that this study will take place in compliance with the general ethical principles applicable to any research involving people.”
Data Analysis
To analyze data we used machine learning algorithms instead of traditional methods, because these algorithms have hyperparameters that can be used to choose the models that best fit the data and optimization parameters to improve prediction capabilities [23, 24]. Machine learning classification and regression algorithms are prediction-designed, but they do not offer inference statistics. Therefore, we resorted to logistic regression to obtain inference information (variable association probability metrics).
To answer RQ1, RQ3, and RQ5, we built machine learning regression models using the random forest (RF) algorithm [23, 24]. Machine learning models are essentially predictive. They are constructed in two phases [23, 24]: the learning stage where the model analyzes and “learn” from the variables associations/relations; and the second stage where the model uses the “learned knowledge” to predict. RF regression models yield, among other outputs, the importance of each predictor variable determined on the basis of a measure called %IncMSE (percent increase in mean squared error). The %IncMSE reflects the increase in MSE (estimated with out-of-bag cross-validation) as a result of each variable being permuted (values randomly shuffled); in simple words, it describes how much (in terms of percentage) the model increases its MSE by excluding each variable. The more the MSE increases, the more important the variable is for the prediction. Thus, the variables can be presented in ranking order of importance. The selection of the most important predictors can be based on the ranking order and in the %IncMSE median value: predictors with %IncMSE values above the median value can be selected as the most important among all set of modeled predictors. RF models are nonparametric; i.e., they do not require a particular structure on the data, and as such they can capture nonlinear relationships, including interactions between the predictors [23, 24].
To answer RQ2, RQ4, and RQ6, we built multinomial logistic regression models because the outcome variables were not normally distributed (they were highly skewed to the left), and therefore they could not be used in linear models. We recoded the outcomes into two categories: low (minimum [min] – first quartile [Q1] values = coded 0) versus intermediate/high (Q2 to maximum values = coded 1).
Some of follow-up measures had missing values. These missing values were handled by the RF algorithm function “rfImpute()” which uses a nearest neighbor machine learning approach either to impute values or to weight their absence [18, 19]. We also used this algorithm to control if the imputation had an effect of the analysis results, which was not the case.
Results
Participants’ Characteristics
Participants were 38.2 years old on average (range 19–75 years, SD = 10.9); most were women (66%); most lived in France (73%); they smoked on average 15.5 cigarettes per day (SD = 7.6); they had been smoking for 19.3 years on average (SD = 11.2); and 59% tested positive on the depression screening test.
Utilization of the App
In the treatment group, the utilization of the app was as follows: different days participants open the app: median = 7, mean = 28; the number of times participants accessed the app: median = 26, mean = 152.87; the interval in days the participant used the app: median = 46, mean = 123.71. In the control group, the utilization of the app was as follows: different days participants open the app: median = 15, mean = 16.84; the number of times participants accessed the app: median = 69, mean = 65.23; the interval in days the participant used the app: median = 87, mean = 90.8.
The Best Predictors of the Number of Different Days when the App Was Accessed
Table 1 presents the machine learning results and lists the 20 variables retained by the algorithm as the best predictors of the number of different days when the app was accessed. As shown in Table 1, among the 20 variables, the %IncMSE ranged from a high of 47.21 (intention to quit smoking assessed at 1-month follow-up) to a low of 0.48 (sex), with a median value of 4.03 (use of heated tobacco product as self-reported after 1-month follow-up). As mentioned, the higher the %IncMSE value, the more important the variable is for successful prediction. In other words, the %IncMSE of a given predictor variable reflects the value of the MSE increase in the prediction model if that variable is removed.
Table 1.
Predictors of the number of different days when the app was accessed, ranked in decreasing order of importance
| Predictors | %IncMSE |
|---|---|
| Intention to quit smoking assessed at 1-month follow-up | 47.21 |
| Cigarettes/day measured at 1-month follow-up | 38.78 |
| Perceived helpfulness of the app measured after 1 month | 22.88 |
| Having quit smoking after 1 month | 17.47 |
| Experiment group (treatment vs. control) | 15.45 |
| Age | 8.94 |
| Number of years smoking | 7.65 |
| Current use of any smoking cessation app, as self-reported at 1-month follow-up | 5.68 |
| Cigarettes/day as measured at baseline | 5.09 |
| Use of e-cigarettes at baseline | 4.56 |
| Use of heated tobacco product as self-reported after 1-month follow-up | 4.03 |
| Number of minutes before the first cigarette of the day | 3.42 |
| Use of nicotine medications after 1-month follow-up | 2.85 |
| Use of e-cigarettes after 1-month follow-up | 2.01 |
| Use of nicotine medications at baseline | 1.71 |
| Depression screening test positive | 1.22 |
| Use/smoke other tobacco products at baseline | 1.14 |
| Country (Switzerland vs. France) | 0.81 |
| Use of heated tobacco product at baseline | 0.58 |
| Sex | 0.48 |
%IncMSE, percent increase in mean squared error, a statistical measure indicating the level on the predictor variable importance in the regression machine learning algorithm.
The 10 most important predictors (those with %IncMSE values above the median) were, in decreasing order of importance, intention to quit smoking assessed after 1 month; number of cigarettes smoked per day measured after 1 month; perceived helpfulness of the app measured after 1 month; having quit smoking after 1 month; experiment group (treatment vs. control); age; number of years smoking; use of any smoking cessation app after 1 month; cigarettes per day measured at baseline; and use of e-cigarettes at baseline. After controlling for the group who used the “dressed down” app, the findings remained statistically the same.
Associations between Each Predictor and the Number of Different Days when the App Was Accessed
Table 2 reports the relationships between each predictor and the number of different days when the app was accessed, a summary of the multinomial logistic regression model. In Table 2, the dichotomic outcome was belonging to the group with intermediate or high number of different days when the app was accessed (“recurrent users,” 3 upper quartiles merged) versus belonging to the group with a low number of different days of use (lowest quartile). The best way to understand Table 2 is to look at the value of coefficients, the sign (negative or positive) associated with the coefficient b, at the corresponding p value, and to take into consideration that the statistical model was designed to predict the probability of a participant belonging to the “recurrent users” class (since the “lowest quartile” class was set as reference class). First, for the largest coefficients, if the p value is inferior to 0.05, that means that the relationship between the relevant predictor variable and the outcome is both substantial and statistically significant. If the sign associated with the coefficient b is positive, that means that an increase in the value of the predictor variable increases the likelihood of belonging to the “recurrent users” class; if the sign is negative, that means that an increase in the predictor values decreases the likelihood of belonging to the “recurrent users” class and increases the likelihood of belonging to the “lowest quartile” class. Thus, the predictors significantly associated with this outcome were: belonging to the control group (vs. treatment group) decreased by 43% the odds of being a recurrent user (odds ratio [OR] = 0.57); a higher number of minutes before the first cigarette (i.e., lower dependence) decreased by 13% per minute the odds of being a recurrent user (OR = 0.87 per minute); the use of any smoking cessation app after 1 month doubled the odds of being a recurrent user; a higher level of intention to quit smoking assessed after 1 month increased by 3.4 times per point on a 3-point scale the odds of being a recurrent user; a higher level of perceived helpfulness of the app measured after 1 month increased by 38% per point the odds of being a recurrent user (OR = 1.38 per point on a 6-point scale); having quit smoking after 1 month increased by 37% the odds of being a recurrent user.
Table 2.
Associations between the predictors and the number of different days when the app was accessed
| Predictors’ category | Predictors | b | OR | SE | p value* | 95% CI | |
|---|---|---|---|---|---|---|---|
| Demographics | Country (Switzerland vs. France) | −0.266 | 0.767 | 0.143 | 0.064 | 0.579 | 1.015 |
| Age | −0.012 | 0.988 | 0.009 | 0.150 | 0.971 | 1.004 | |
| Sex | −0.217 | 0.805 | 0.132 | 0.100 | 0.622 | 1.043 | |
| Treatment/control | Experiment group | −0.564 | 0.569 | 0.124 | <0.001 | 0.446 | 0.726 |
| Baseline | Number of years smoking | −0.008 | 0.992 | 0.008 | 0.352 | 0.976 | 1.009 |
| Cigarettes/day | −0.016 | 0.984 | 0.010 | 0.105 | 0.964 | 1.003 | |
| Number of minutes before the first cigarette of the day | −0.142 | 0.868 | 0.064 | 0.026 | 0.766 | 0.983 | |
| Use/smoke other tobacco products | −0.006 | 0.994 | 0.054 | 0.918 | 0.895 | 1.105 | |
| Use of heated tobacco products | 0.114 | 1.120 | 0.135 | 0.401 | 0.859 | 1.461 | |
| Use of e-cigarettes | −0.040 | 0.961 | 0.065 | 0.540 | 0.846 | 1.091 | |
| Use of nicotine medications | −0.101 | 0.904 | 0.184 | 0.584 | 0.631 | 1.297 | |
| Depression | 0.134 | 1.144 | 0.123 | 0.276 | 0.898 | 1.456 | |
| Follow-up after 1 month | Cigarettes/day | 0.020 | 1.021 | 0.013 | 0.126 | 0.994 | 1.048 |
| Use of heated tobacco product | −0.517 | 0.596 | 0.275 | 0.060 | 0.348 | 1.022 | |
| Use of e-cigarettes | −0.156 | 0.856 | 0.091 | 0.087 | 0.716 | 1.023 | |
| Use of nicotine medications | 0.048 | 1.049 | 0.161 | 0.767 | 0.764 | 1.440 | |
| Current use of any smoking cessation app | 0.785 | 2.193 | 0.132 | <0.001 | 1.691 | 2.843 | |
| Intention to quit smoking | 1.236 | 3.443 | 0.150 | <0.001 | 2.565 | 4.623 | |
| Perceived helpfulness of the app | 0.321 | 1.378 | 0.063 | <0.001 | 1.218 | 1.559 | |
| Having quit smoking | −0.472 | 0.624 | 0.143 | <0.001 | 0.471 | 0.826 | |
b, beta coefficient; OR, odds ratio; SE, standard error; CI, confidence interval.
*Significant at least at p < 0.05.
The Best Predictors of the Number of Times the App Was Opened
Table 3 displays the machine learning results and lists the 20 best predictors of the number of times the app was opened. As shown in Table 3, among the 20 variables, the %IncMSE ranged from a high of 44.31 (intention to quit smoking assessed at 1-month follow-up) to a low of 0.03 (sex), with a median value of 3.12 (use of nicotine medications after 1 month).
Table 3.
Predictors of the number of times the app was opened, ranked in decreasing order of importance
| Predictors | %IncMSE |
|---|---|
| Intention to quit smoking assessed after 1 month | 44.31 |
| Cigarettes/day measured at 1-month follow-up | 39.96 |
| Perceived helpfulness of the app measured after 1 month | 27.09 |
| Experiment group (treatment vs. control) | 25.89 |
| Having quit smoking after 1-month follow-up | 14.97 |
| Current use of any smoking cessation app, as self-reported at 1-month follow-up | 9.98 |
| Number of years smoking | 9.72 |
| Age | 8.31 |
| Cigarettes/day measured at baseline | 5.94 |
| Number of minutes before the first cigarette of the day | 3.63 |
| Use of nicotine medications after 1 month | 3.12 |
| Depression | 3.08 |
| Use of heated tobacco product as self-reported after 1 month | 2.99 |
| Use of e-cigarettes at baseline | 2.44 |
| Use of e-cigarettes as self-reported after 1 month | 2.23 |
| Country (Switzerland vs. France) | 0.77 |
| Use of nicotine medications at baseline | 0.61 |
| Use/smoke other tobacco products at baseline | 0.57 |
| Use of heated tobacco product as self-reported after 1-month follow-up | 0.49 |
| Sex | 0.03 |
%IncMSE, percent increase in mean squared error, a statistical measure indicating the level on the predictor variable importance in the regression machine learning algorithm.
The 10 best predictors (those with %IncMSE values above the median) were, in decreasing order of importance, intention to quit smoking assessed after 1 month; cigarettes per day measured after 1 month; perceived helpfulness of the app measured after 1 month; experiment group (vs. control); having quit smoking at 1 month; use of any smoking cessation app as self-reported after 1 month; number of years smoking; age; cigarettes per day; and number of minutes before the first cigarette of the day measured at baseline. After controlling for the group who used the “dressed down” app, the findings remained statistically the same.
Associations between Each Predictor and the Number of Times the App Was Opened
Table 4 displays the relationships between each predictor and the number of times the app was opened. In this table, the dichotomic outcome was belonging to the group with intermediate or high number of times when the app was opened (“frequent users”) versus belonging to the group with a low number of times when the app was opened. The predictors significantly associated with this outcome were: belonging to the control group (vs. treatment group) decreased by 60% the odds of being a frequent user (OR = 0.401); a higher number of cigarettes/day as measured at baseline decreased by 3% per cig./day the odds of being a frequent user (OR = 0.97 per cig./day); a higher number of minutes before the first cigarette decreased by 13% per minute the odds of being a frequent user (OR = 0.87 per minute); a higher number of cigarettes/day as measured after 1 month decreased by 4% per cig./day the odds of being a frequent user (OR = 1.04); the use of any smoking cessation app after 1-month follow-up doubled the odds of being a frequent user; a higher level of intention to quit smoking assessed after 1 month doubled the odds of being a frequent user; a higher perceived helpfulness of the app measured after 1 month increased by 44% per point the odds of being a frequent user (OR = 1.44 per point on a 6-point scale); having quit smoking after 1 month decreased by 44% the odds of being a frequent user (OR = 0.56).
Table 4.
Associations between the predictors and the number of times the app was opened
| Predictors’ category | Predictors | b | OR | SE | p value* | 95% CI | |
|---|---|---|---|---|---|---|---|
| Demographics | Country (Switzerland vs. France) | −0.117 | 0.890 | 0.146 | 0.424 | 0.668 | 1.184 |
| Age | −0.007 | 0.993 | 0.009 | 0.420 | 0.976 | 1.010 | |
| Sex | −0.242 | 0.785 | 0.135 | 0.073 | 0.603 | 1.023 | |
| Treatment/control | Experiment group | −0.914 | 0.401 | 0.130 | <0.001 | 0.311 | 0.518 |
| Baseline | Number of years smoking | 0.000 | 1.000 | 0.009 | 0.959 | 0.983 | 1.018 |
| Cigarettes/day | 0.033 | 0.968 | 0.011 | 0.002 | 0.948 | 0.988 | |
| Number of minutes before the first cigarette of the day | −0.139 | 0.870 | 0.065 | 0.034 | 0.765 | 0.989 | |
| Use/smoke other tobacco products | −0.028 | 0.973 | 0.055 | 0.617 | 0.873 | 1.084 | |
| Use of heated tobacco product | 0.099 | 1.104 | 0.138 | 0.474 | 0.842 | 1.446 | |
| Use of e-cigarettes | −0.052 | 0.949 | 0.067 | 0.433 | 0.833 | 1.081 | |
| Use of nicotine medications | −0.261 | 0.770 | 0.190 | 0.169 | 0.531 | 1.117 | |
| Depression | 0.036 | 1.037 | 0.126 | 0.774 | 0.810 | 1.328 | |
| Follow-up after 1 month | Cigarettes/day | 0.038 | 1.039 | 0.013 | 0.005 | 1.012 | 1.067 |
| Use of heated tobacco product | −0.317 | 0.728 | 0.278 | 0.254 | 0.423 | 1.255 | |
| Use of e-cigarettes | −0.089 | 0.915 | 0.094 | 0.344 | 0.762 | 1.100 | |
| Use of nicotine medications | 0.198 | 1.219 | 0.169 | 0.240 | 0.876 | 1.696 | |
| Current use of any smoking cessation app | 0.763 | 2.145 | 0.138 | <0.001 | 1.637 | 2.811 | |
| Intention to quit smoking | 0.810 | 2.247 | 0.149 | <0.001 | 1.679 | 3.008 | |
| Perceived helpfulness of the app | 0.363 | 1.438 | 0.066 | <0.001 | 1.263 | 1.637 | |
| Having quit smoking | −0.582 | 0.559 | 0.150 | <0.001 | 0.416 | 0.750 | |
b, beta coefficient; OR, odds ratio; SE, standard error; CI, confidence interval.
*Significant at least at p < 0.05.
The Best Predictors of the Apps Use Duration
Table 5 exhibits the machine learning results and lists the best 22 predictors of the app use duration (i.e., interval in days between first and last use). The 10 most important predictors were, in decreasing order of importance, cigarettes/day measured after 1 month; intention to quit smoking assessed after 1 month; perceived helpfulness of the app measured after 1 month; having quit smoking after 1 month; use of any smoking cessation app as self-reported at 1 month; age; number of years smoking; experimental group (vs. control group); use of nicotine medications at 1 month; and minutes before the first cigarette measured at baseline.
Table 5.
The importance of each predictor of the apps use duration: decreasing order
| Predictors | %IncMSE |
|---|---|
| Cigarettes/day measured at 1-month follow-up | 135.43 |
| Intention to quit smoking assessed after 1-month follow-up | 125.52 |
| Perceived helpfulness of the app measured after 1-month follow-up | 56.59 |
| Having quit smoking after 1-month follow-up | 44.66 |
| Current use of any smoking cessation app, as self-reported at 1-month follow-up | 28.23 |
| Age | 26.98 |
| Number of years smoking | 22.02 |
| Experiment group (treatment vs. control) | 20.01 |
| Use of nicotine medications after 1 month | 15.68 |
| Number of minutes before the first cigarette of the day | 14.86 |
| Use of nicotine medications at baseline | 12.87 |
| Use of heated tobacco product as self-reported after 1-month follow-up | 12.67 |
| Cigarettes/day measured at baseline | 9.54 |
| Sex | 8.51 |
| Depression screening test positive | 7.92 |
| Country (Switzerland vs. France) | 6.76 |
| Use of e-cigarettes at baseline | 4.21 |
| Use/smoke other tobacco products at baseline | 3.51 |
| Use of e-cigarettes after 1-month follow-up | 1.57 |
| Use of heated tobacco product at baseline | 0.61 |
%IncMSE, percent increase in mean squared error, a statistical measure indicating the level on the predictor variable importance in the regression machine learning algorithm.
Associations between Each Predictor and the Apps Use Duration
Table 6 displays the relationships between each predictor variable and the app use duration. In this table, the dichotomic outcome was belonging to the group with intermediate or high duration of app use (“long-term users”) versus belonging to the group with low duration of use. The predictors significantly associated with this outcome were: living in France (vs. in Switzerland) decreased by 34% the odds of being a long-term user; older age decreased by 2% per year the likelihood of being a long-term user (OR = 0.98 per year of age); belonging to the control group (vs. treatment group) decreased by 39% the odds of being a long-term user; a higher number of cigarettes/day measured after 1 month increased the likelihood of being a long-term user (OR = 1.05 per cig/day); the use of any smoking cessation app after 1 month increased by 85% the odds of being a long-term user; intention to quit smoking assessed after 1 month increased by 2.5 times per point the odds of being a long-term user (OR = 2.5 per point on a 3-point scale); perceiving the app as helpful after 1 month increased by 40% per point the odds of being a long-term user (OR = 1.4 per point on a 6-point scale).
Table 6.
Associations between the predictors and the apps use duration
| Predictors’ category | Predictors | b | OR | SE | p value* | 95% CI | |
|---|---|---|---|---|---|---|---|
| Demographics | Country (Switzerland vs. France) | −0.411 | 0.663 | 0.147 | 0.005 | 0.497 | 0.885 |
| Age | 0.019 | 0.981 | 0.009 | 0.031 | 0.964 | 0.998 | |
| Sex | −0.040 | 0.961 | 0.134 | 0.764 | 0.739 | 1.249 | |
| Treatment/control | Experiment group | −0.495 | 0.610 | 0.126 | <0.001 | 0.476 | 0.780 |
| Baseline | Number of years smoking | 0.001 | 1.001 | 0.009 | 0.934 | 0.983 | 1.018 |
| Cigarettes/day as measured | −0.009 | 0.991 | 0.010 | 0.391 | 0.972 | 1.011 | |
| Number of minutes before the first cigarette of the day | −0.112 | 0.894 | 0.064 | 0.081 | 0.787 | 1.014 | |
| Use/smoke other tobacco products | −0.001 | 0.999 | 0.055 | 0.985 | 0.898 | 1.112 | |
| Use of heated tobacco product | 0.078 | 1.081 | 0.140 | 0.576 | 0.822 | 1.423 | |
| Use of e-cigarettes | −0.094 | 0.910 | 0.066 | 0.154 | 0.799 | 1.036 | |
| Use of nicotine medications | −0.049 | 0.952 | 0.185 | 0.791 | 0.662 | 1.369 | |
| Depression | 0.184 | 1.202 | 0.125 | 0.141 | 0.941 | 1.536 | |
| Follow-up after 1 month | Cigarettes/day | 0.045 | 1.046 | 0.013 | <0.001 | 1.018 | 1.073 |
| Use of heated tobacco product | 0.220 | 1.246 | 0.290 | 0.449 | 0.705 | 2.200 | |
| Use of e-cigarettes | 0.049 | 1.050 | 0.094 | 0.603 | 0.874 | 1.261 | |
| Use of nicotine medications | 0.043 | 1.044 | 0.163 | 0.791 | 0.759 | 1.437 | |
| Current use of any smoking cessation app | 0.614 | 1.847 | 0.135 | <0.001 | 1.418 | 2.407 | |
| Intention to quit smoking | 0.916 | 2.500 | 0.148 | <0.001 | 1.869 | 3.345 | |
| Perceived helpfulness of the app | 0.340 | 1.404 | 0.065 | <0.001 | 1.237 | 1.595 | |
| Having quit smoking | −0.249 | 0.779 | 0.146 | 0.087 | 0.586 | 1.037 | |
b, beta coefficient; OR, odds ratio; SE, standard error; CI, confidence interval.
*Significant at least at p < 0.05.
Discussion
The Best Predictors of Utilization of the App
Overall, the machine learning algorithms revealed that the best predictors (%IncMSE >5 in both Tables 2 and 3) of the participants’ level of engagement with this smoking cessation app were intention to stop smoking, dependence level (cigarettes per day and minutes to the first cigarette), perceived helpfulness of the app, having quit smoking after 1 month, self-reported usage of the smoking cessation app after 1 month, belonging to the experimental group (vs. control group), age and duration (years) of smoking.
Associations with the App Use
Participants declaring higher intention to quit smoking after 1 month were more likely to be repeated users of the app than those who reported lower levels of motivation to quit. Similarly, participants who had quit smoking after 1 month were more likely to be repeated users of the app than those who failed to quit. This suggests that motivation to quit smoking and being actively involved in a quit attempt were major drivers to use the Stop-Tabac app. In contrast, failing to quit smoking may lead to disengagement. The results are concordant with the COMB model [16], highlighting the importance of the motive components in behavior change. The role of initial motivation for treatment maintenance was already described for addictive disorders in different clinical settings [25]. These results can be used to better target the app at smokers who are the most motivated to quit or the most actively engaged in a quit attempt, but they can also be used to design strategies to regain the attention and the participation of those who fail to quit, who are discouraged and disengaged and have lost interest in the app or in smoking cessation.
Associations with cig/day were inconclusive (the associations were not consistently in the same direction), but higher nicotine dependence as reflected by a shorter time before smoking the first cigarette of the day was consistently associated with using the app more frequently, but the present study data do not enable us to explain why the two indicators of dependence (cig/day and minutes) did not perform similarly. Previous research on this association is also inconclusive, as some reports found that heavier smokers had lower utilization of another smoking cessation app [3] while others found no association [10]. Thus, the association between level of cigarette dependence and level of engagement with smoking cessation apps needs to be further investigated.
Participants who after 1 month reported that the app was helpful were more likely to be repeated users (which is congruent with a previous review [14]), as were participants in the experimental group, most probably because participants in the Stop-Tabac group were more satisfied with the app than participants in the control group. Older participants were more likely than younger participants to be frequent users of the app, which is congruent with a previous report [11]. This is in contrast with the common view that older people are not at ease with technology [26]. This finding is possibly explained because the app was not designed specifically for young users nor with them. This could be also partly explained by the age trends for smoking cessation [27]. Further developments of the app may specifically target older smokers, and further studies are warranted to adapt smartphone apps for young people [28].
Many women (66%) enrolled in our study, but contrary to previous research we found no associations between gender and the utilization of the app [3, 11]. The rate of people screening positively for depression (59%) was higher than in other studies of smokers [29]. In contrast with other studies [30], we did not find any association between depression and utilization of the app.
While in previous study [6] the full Stop-Tabac app was not significantly more efficient for smoking cessation than the control version app, the fact that, in present study, participants in the treatment group (vs. control group) were significantly more likely to use the smoking cessation app seems to confirm the relative usefulness of the features included in the full Stop-Tabac app compared to the control version. It is important to note that statistically the predictors of participants’ engagement with the app were the same in both study groups, which means that when controlling for the group belonging the effect of the other predictors did not significantly change. These two observations combined might mean that an improvement of this kind of apps in terms of design and tailoring to specific smokers may lead to significant efficacity.
Limitation
As this study was not initially planned for this secondary data analysis, the data set included a limited number of predictor variables. In particular, we did not measure education level, a predictor of lower app utilization in previous studies [3]. The study also lacks information about cultural or ethnic background as well as on socio-economic status of the participants, which may impact health literacy and health-related behaviors [31]. The response rate at follow-up was low, but we included a large sample of people of all ages who lived in all the departments of France and all the cantons of French-speaking Switzerland, and we collected data on the utilization of the app in 100% of participants.
Conclusion
More frequent use of a smoking cessation app for smartphones was predicted mainly by higher levels of motivation to quit smoking, by having recently quit smoking, by higher levels of nicotine dependence, by the perceived helpfulness of the app, and by an older age of participants. This information can be used to further target and adapt the apps to the needs of these specific groups of users (i.e., smokers highly motivated to quit, older and more dependent smokers), to develop strategies to enroll and retain smokers (e.g., those who are ambivalent about quitting smoking, younger and less dependent smokers). Assessing whether the target audience is satisfied with the app and perceives it as useful seems to be important to ensure that the apps’ content fits the needs of users and to ensure smokers engage with the app and use it regularly.
Acknowledgments
We would like to thank Vincent Baujard and Evelyne Laszlo for their valuable contributions to the development of the Stop-Tabac smartphone application in which the randomized trial was based.
Statement of Ethics
The study was submitted to the Cantonal Ethics Committee in Geneva (Req-2018-00356) who answered that, according to Swiss law, the app being no medical device, the study did not legally need approval from this commission. Therefore, the commission did not formally review our proposal but answered that “everything indicates that this study will take place in compliance with the general ethical principles applicable to any research involving people” (e-mail dated May 16, 2018; address: Commission Cantonale d’Ethique de la Recherche [CCER], rue Adrien-Lachenal 8, 1207 Geneva, Switzerland). No consent was needed for publication. Clinical trial registration: ISRCTN Registry: ISRCTN11318024, May 17, 2018, http://www.isrctn.com/ISRCTN11318024. Written informed consent was obtained from participants to participate in the study.
Conflict of Interest Statement
The authors have no conflicts of interest to declare.
Funding Sources
This study was supported by the Swiss National Science Foundation, Grant 32 003-179 369, CHF 194942 (EUR 182200, USD 200700). The authors did not receive any other funding or sponsorship; their salaries were paid by the universities/researcher institutions they work for. The study funding source did not play any role in the preparation of data or in the preparation of manuscript.
Author Contributions
G.V.C., J.F.E., and Y.K. conceptualized the study and drafted the manuscript. G.V.C. conducted the data analysis. J.F.E. ran the data collection and implementation of the intervention, obtained resources, and supervised assistants. Y.K. worked on funding acquisition.
Funding Statement
This study was supported by the Swiss National Science Foundation, Grant 32 003-179 369, CHF 194942 (EUR 182200, USD 200700). The authors did not receive any other funding or sponsorship; their salaries were paid by the universities/researcher institutions they work for. The study funding source did not play any role in the preparation of data or in the preparation of manuscript.
Data Availability Statement
The material used in this study and data supporting this finding are available at https://archive.org/details/@stopdependance_ch. The original data are available in open sources at Yareta: https://yareta.unige.ch/home/detail/d8d9b746-3fe4-4077-a4f3-552154c16699. Further inquiries can be directed to the corresponding author.
Supplementary Material
References
- 1. Ezzati M, Lopez AD. Estimates of global mortality attributable to smoking in 2000. Lancet. 2003;362(9387):847–52. 10.1016/S0140-6736(03)14338-3. [DOI] [PubMed] [Google Scholar]
- 2. Whittaker R, McRobbie H, Bullen C, Rodgers A, Gu Y, Dobson R. Mobile phone text messaging and app-based interventions for smoking cessation. Cochrane Database Syst Rev. 2019;10:CD006611. 10.1002/14651858.CD006611.pub5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Patel R, Sulzberger L, Li G, Mair J, Morley H, Shing MNW, et al. Smartphone apps for weight loss and smoking cessation: quality ranking of 120 apps. N Z Med J. 2015;128(1421):73–6. [PubMed] [Google Scholar]
- 4. Chu KH, Matheny SJ, Escobar-Viera CG, Wessel C, Notier AE, Davis EM. Smartphone health apps for tobacco Cessation: a systematic review. Addict Behav. 2021;112:106616. 10.1016/j.addbeh.2020.106616. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Barnett A, Ding H, Hay KE, Yang IA, Bowman RV, Fong KM, et al. The effectiveness of smartphone applications to aid smoking cessation: a meta-analysis. Clin eHealth. 2020;3:69–81. 10.1016/j.ceh.2020.09.001. [DOI] [Google Scholar]
- 6. Etter JF, Khazaal Y. The Stop-tabac smartphone application for smoking cessation: a randomized controlled trial. Addiction. 2022;117(5):1406–15. 10.1111/add.15738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Thornton L, Quinn C, Birrell L, Guillaumier A, Shaw B, Forbes E, et al. Free smoking cessation mobile apps available in Australia: a quality review and content analysis. Aust N Z J Public Health. 2017;41(6):625–30. 10.1111/1753-6405.12688. [DOI] [PubMed] [Google Scholar]
- 8. Zeng EY, Vilardaga R, Heffner JL, Mull KE, Bricker JB. Predictors of utilization of a novel smoking cessation smartphone app. Telemed J E Health. 2015;21(12):998–1004. 10.1089/tmj.2014.0232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Bricker JB, Mull KE, Santiago-Torres M, Miao Z, Perski O, Di C. Smoking cessation smartphone app use over time: predicting 12-month cessation outcomes in a 2-arm randomized trial. J Med Internet Res. 2022;24(8):e39208. 10.2196/39208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Iacoviello BM, Steinerman JR, Klein DB, Silver TL, Berger AG, Luo SX, et al. Clickotine, a personalized smartphone app for smoking cessation: initial evaluation. JMIR Mhealth Uhealth. 2017;5(4):e56. 10.2196/mhealth.7226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Becker S, Kribben A, Meister S, Diamantidis CJ, Unger N, Mitchell A. User profiles of a smartphone application to support drug adherence: experiences from the iNephro project. PLoS One. 2013;8(10):e78547. 10.1371/journal.pone.0078547. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Peiris D, Wright L, News M, Rogers K, Redfern J, Chow C, et al. A smartphone app to assist smoking cessation among Aboriginal Australians: findings from a pilot randomized controlled trial. JMIR Mhealth Uhealth. 2019;7(4):e12745. 10.2196/12745. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Garrison K, Pal P, O’Malley S, Pittman B, Gueorguieva R, Rojiani R, et al. Craving to Quit: a randomised controlled trial of smartphone app-based mindfulness training for smoking cessation. Nicotine Tob Res. 2020 Mar 16;22(3):324–31. 10.1093/ntr/nty126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Szinay D, Jones A, Chadborn T, Brown J, Naughton F. Influences on the uptake of and engagement with health and well-being smartphone apps: systematic review. J Med Internet Res. 2020;22(5):e17572. 10.2196/17572. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Ng JYY, Ntoumanis N, Thogersen-Ntoumani C, Deci EL, Ryan RM, Duda JL, et al. Self-determination theory applied to health contexts: a meta-analysis. Perspect Psychol Sci. 2012;7(4):325–40. 10.1177/1745691612447309. [DOI] [PubMed] [Google Scholar]
- 16. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. 10.1186/1748-5908-6-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Monney GL, Penzenstadler L, Dupraz O, Etter JF, Khazaal Y. mHealth app for cannabis users: satisfaction and perceived usefulness. Front Psychiatry. 2015;6:120. 10.3389/fpsyt.2015.00120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Livingstone-Banks J, Norris E, Hartmann-Boyce J, West R, Jarvis M, Chubb E, et al. Relapse prevention interventions for smoking cessation. Cochrane Database Syst Rev. 2019;2019(10):CD003999. 10.1002/14651858.CD003999.pub6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Garnett C, Crane D, West R, Brown J, Michie S. Identification of behavior change techniques and engagement strategies to design a smartphone app to reduce alcohol consumption using a formal consensus method. JMIR Mhealth Uhealth 2015;3(2):e73. 10.2196/mhealth.3895. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Seo S, Cho SI, Yoon W, Lee CM. Classification of smoking cessation apps: quality review and content analysis. JMIR mhealth uhealth. 2022;10(2):e17268. 10.2196/17268. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Etter JF, Khazaal Y. The Stop-Tabac smartphone application for smoking cessation: study protocol for a randomized controlled trial in the general population. Trials. 2020;21(1):449. 10.1186/s13063-020-04377-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Whooley MA, Avins AL, Miranda J, Browner WS. Case-finding instruments for depression. Two questions are as good as many. J Gen Intern Med. 1997;12(7):439–45. 10.1046/j.1525-1497.1997.00076.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Breiman L. Random forests. Mach Learn. 2001;45(1):5–32. 10.1023/a:1010933404324. [DOI] [Google Scholar]
- 24. Breiman L. Manual on setting up, using, and understanding random forests, V3.1. 2002. Available from: https://www.stat.berkeley.edu/∼breiman/Using_random_forests_v4.0.pdf. [Google Scholar]
- 25. Ryan RM, Plant RW, O’Malley S. Initial motivations for alcohol treatment: relations with patient characteristics, treatment involvement, and dropout. Addict Behav. 2015;20(3):279–97. 10.1016/0306-4603(94)00072-7. [DOI] [PubMed] [Google Scholar]
- 26. Askari M, Klaver NS, van Gestel TJ, van de Klundert J. Intention to use medical apps among older adults in The Netherlands: cross-sectional study. J Med Internet Res. 2020;22(9):e18080. 10.2196/18080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Pesce G, Marcon A, Calciano L, Perret JL, Abramson MJ, Bono R, et al. Time and age trends in smoking cessation in Europe. PLoS One. 2019;14(2):e0211976. 10.1371/journal.pone.0211976. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Kenny R, Dooley B, Fitzgerald A. Developing mental health mobile apps: exploring adolescents’ perspectives. Health Inform J. 2016;22(2):265–75. 10.1177/1460458214555041. [DOI] [PubMed] [Google Scholar]
- 29. Hebert KK, Cummins SE, Hernández S, Tedeschi GJ, Zhu SH. Current major depression among smokers using a state quitline. Am J Prev Med. 2011;40(1):47–53. 10.1016/j.amepre.2010.09.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Molloy A, Anderson PL. Engagement with mobile health interventions for depression: a systematic review. Internet Interv. 2021;26:100454. 10.1016/j.invent.2021.100454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. von dem Knesebeck O, Mnich E, Daubmann A, Wegscheider K, Angermeyer MC, Lambert M, et al. Socioeconomic status and beliefs about depression, schizophrenia and eating disorders. Soc Psychiatry Psychiatr Epidemiol. 2013;48(5):775–82. 10.1007/s00127-012-0599-1. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The material used in this study and data supporting this finding are available at https://archive.org/details/@stopdependance_ch. The original data are available in open sources at Yareta: https://yareta.unige.ch/home/detail/d8d9b746-3fe4-4077-a4f3-552154c16699. Further inquiries can be directed to the corresponding author.