Abstract
Background
Text messaging interventions have shown promise in helping people quit smoking. Texting programs periodically survey participants about their smoking status. This study examined the consistency of participant self-reported smoking between external surveys and internal program text message assessments.
Methods
Participants in Text2Quit program were surveyed about their past 7-day smoking at one, three, and six months post-enrollment using different survey modes (external surveys and internal program text message assessments) and responses were compared for consistency. The first set of analyses was conducted for participants responding on both modes (n = 45 at one month; n = 50 at three months; n = 42 at six months). Additional analyses, assuming missing = smoking, were conducted with the full sample of 262 smokers (68.7% female, mean age = 35.8 years) and compared to saliva-confirmed abstinence rates.
Results
Participants responding to both modes consistently reported smoking status at one (88.9%), three (88.0%) and six (88.1%) months post-enrollment, with fair to substantial levels of agreement (one month: κ = 0.24; three months: κ = 0.63; six months: κ = 0.66). Participants responding to both modes reported high rates of abstinence. In missing = smoking analyses, significant differences in abstinence rates reported across modes were detected at each timepoint (one month: external = 30.5%, internal = 16.4%; three months: external = 33.2%, internal = 16.0%; six months: external = 31.7%, internal = 12.2%; all p < .001). Moderate levels of agreement were found between the two modes. At 6 months, abstinence rates obtained via internal data were closer to those biochemically verified (15.7%) compared to external surveys.
Conclusions
Results provide initial support for the use of internal program assessments in text messaging programs with missing = smoking assumptions in order to gather outcome data on smoking behavior.
Keywords: Smoking cessation, Text messaging, mHealth, Outcomes, Assessment
1. Introduction
Given the fast pace of technological change, timely evaluations of technology-based behavior change programs are needed. Over the past several years, a growing body of evidence has accumulated in support of the efficacy of automated text messaging programs for health behavior change including smoking cessation (Free et al., 2013; Whittaker, McRobbie, Bullen, Rodgers, & Gu, 2016; Whittaker, Merry, Dorey, & Maddison, 2012). A recent Cochrane Review of 12 such studies concluded that these mobile intervention double the chances of long-term quitting compared to a control condition (Whittaker et al., 2016). With increasing evidence to support their use, adoption of these scalable programs has increased (Abroms, Carroll, Boal, Mendel, & Carpenter, 2016; ITU, 2017; NCD Alliance, 2017).
Typically, evaluations of mHealth programs have relied on data that are collected through a separate survey designed for evaluation (referred to in this paper as “external survey data”) through in-person, phone, or web-surveys conducted by the research team, rather than outcomes captured as part of the intervention (referred to as “internal program data”) (Abroms, Boal, Simmens, Mendel, & Windsor, 2014; Free et al., 2011; Whittaker et al., 2016; Ybarra, Jiang, Free, Abroms, & Whittaker, 2016). However, the collection of external survey data to track study outcomes and effectiveness is time and resource intensive. Thus, different and scalable methods to assess program outcomes are needed.
One way to achieve timelier and less resource intensive evaluations is to make use of data that are collected as part of a program. Internal program data are generally collected in routine intervention delivery, often to enhance or customize the intervention, and may include surveys that are automated and repeated over time. In some cases, programs may make use of the same (or a very similar) assessment questions as those used in external surveys. Internal program data may offer the advantages of being collected automatically, being significantly cheaper to collect than external surveys, and providing information on a real-time basis (Riley et al., 2011). In fact, some studies are already relying exclusively on internal program data to evaluate mobile apps for smoking cessation (Ubhi, Michie, Kotz, Wong, & West, 2015) and text messaging interventions (Christofferson, Hertzberg, Beckham, Dennis, & Hamlett-Berry, 2016; Cole-Lewis et al., 2016).
In contrast to the potential advantages of using internal program data for evaluation, these data may be subject to a number of limitations, including low response rates and the response bias that may come from participant knowledge that survey data are used to enhance or alter the ongoing intervention. Perhaps most concerning are the low response rates for internal program assessments. In the case of text messaging programs for smoking cessation, previous studies have found response rates for messages to assess abstinence ranging from 34% for the 7-day post quit day assessment to as low as 4% at the 6-month post-treatment assessment in routine intervention delivery (Cole-Lewis et al., 2016). Response rates of between 74% and 62% have been found in clinical trials, with a general pattern of decreasing rates over time (Naughton, Riaz, & Sutton, 2016).
As efforts are underway to roll out text messaging programs in low and middle income countries to address non-communicable diseases such as those caused by tobacco use (ITU, 2017; NCD Alliance, 2017), studies are needed that investigate the effectiveness of these programs. Moreover, as these programs may be adapted and modified over the course of their implementation, studies are needed that investigate their effectiveness on an ongoing basis. In the absence of external survey data, internal program data may prove informative, but studies are needed to investigate the concordance between these assessment modes.
Using data from the Text2Quit study, a RCT of a text messaging program for smoking cessation (Abroms et al., 2014), the current study examines if internal program data, collected via text message, can provide a reliable surrogate for external survey data. Specifically, the study seeks to answer two research questions. 1) What is the concordance in responses of smoking status between those who responded to the external survey and to the internal program text message assessments? 2) What is the concordance in quit rates between the assessment modes when assuming missing = smoking?
2. Methods
2.1. Study procedures and intervention
The study was approved by the George Washington University (GWU) Institutional Review Board (IRB #040810). Recruitment occurred between May 19, 2011 and July 10, 2012. Recruitment and enrollment took place on the Internet through Google keyword search. Participants who searched Google with keywords related to quitting smoking were displayed study recruitment advertisements in conjunction with their search results. Clicking on ads directed participants to the study website with a screening survey and informed consent procedures. Participants were at least 18 years of age, smoked 5 or more cigarettes a day, had a cell phone number with an unlimited SMS plan, and had an interest in quitting smoking in the next month. This study examines the smoking status of the 262 participants randomized to the intervention group of an RCT of the Text2Quit program. Text messages provided participants with the opportunity to opt out and 30.1% of participants (n = 79) used the keyword STOP to unsubscribe from program texts during the 6-month intervention period. Additional details about the study are reported elsewhere (Abroms et al., 2014).
2.2. Data collection
Data were used from the text messaging program Text2Quit where internal program assessments automatically delivered via text message made use of the same survey question as the external surveys studying the Text2Quit intervention. Text2Quit is a text message-based program, which sends automated, interactive and tailored text messages timed around a subscriber’s quit date to aid in smoking cessation. In an external surveys, those randomized to Text2Quit were found at the 6 month follow-up to self-report quitting at higher rates compared to the control group for past 7 day smoking (RR = 1.53, 95% CI = 1.13–2.07, p < .01), (Abroms et al., 2014). As part of the program evaluation, participants received external surveys that assessed, among other items, smoking over the past 7 days. In addition, as part of the Text2Quit program, participants received periodic text message surveys, which assessed past 7-day smoking.
External surveys were predominantly self-administered by study participants through Survey Monkey, but were also done by phone, text, or email where needed. The external surveys were given at 1, 3 and 6-month post-enrollment. Up to 10 attempts were made to remind participants to fill out their external surveys. Participants received a $15 Amazon gift card for each survey completed. The vast majority of participants completed the external survey online (1 month, 91.0%, 3 months, 90.2%; 6 months, 91.1%) with the remaining by phone, text, or e-mail (Abroms et al., 2014).
Program text messages were sent every 7 days in the first month following a participant’s quit date and then at 60 days, 90 days, and 6-month after their quit date. After a participant completed the assessment, they received tailored replies (e.g., encouragement and praise, strategies to cope with slips and relapses) and additional help if they indicated that they had smoked. Data of text responses to internal program assessments were recorded on computer servers maintained by the Text2Quit service provider, Voxiva Inc.
2.3. Measures
The primary outcome of interest was self- reported 7-day point prevalence abstinence (Hughes et al., 2003) at 1, 3 and 6-month post-enrollment as reported to external survey (“Have you smoked a cigarette, even a puff, in the last 7 days?”) and post-quit date as reported via internal program text message (“Over the past 7 days, have you smoked a cig, even a drag?”). Participants were considered abstinent, if they reported not smoking in the past 7 days. Saliva was collected by mail from participants who reported not smoking in the past 7 days at the 6-month follow-up and cotinine levels of ≤15 ng/mL were considered abstinent (65.1% completion rate) (Abroms et al., 2014).
2.4. Sample
To be included in the first analysis (Research Question 1), an individual had to report their smoking status to both external survey and internal program assessments within 14 days pre/post the external survey date.
The second analysis (Research Question 2, missing = smoking) included the entire sample of 262 people assigned to receiving the Text2Quit program. Non-responders on either assessment mode were coded as still smoking.
2.5. Analysis
Research Question 1
Smoking status responses across the two modes (external survey and internal program data) were compared for response consistency. Percent agreement was calculated. McNemar’s test compared the marginal proportion of consistent reports to external survey and internal program assessments and was chosen because it accounts for the dependency of data with multiple assessments per participant. Cohen’s Kappa, a measure of chance-corrected agreement, was used to comparatively assess how well the two modes were able to ascertain an individual’s self-reported smoking status (Viera & Garrett, 2005).
Research Question 2
The second set of analyses compared smoking status across both modes assuming non-response as smoking (missing = smoking) for both assessment modes. Additional analyses compared self-reported and biochemically verified smoking rates. Again, we calculated percent agreement, McNemar’s tests, and Cohen’s Kappa.
3. Results
3.1. Sample characteristics
Of the 262 participants, 6 responded to the internal program assessments only (2.3%), 108 responded to the external surveys only (41.2%), and 124 responded to both assessment types at any point during the follow-up phase (47.3%). The mean number of internal program assessments completed for all 130 participants (49.6%) who completed an internal program assessment was 2.12 (SD = 2.88, range 0–11). External surveys were completed by 232 participants (88.6%) and at a slightly higher rate (2.35 surveys per participant, SD = 1.04, range 0–3). A total of 88 participants (33.6%) were classified as “dual-mode” responders, who completed an external survey and internal program assessment within 14 days pre/post the external survey date. Compared to non-dual mode responders, ever dual mode responders were more likely to be female, had higher education, smoked fewer cigarettes per day, and reported lower nicotine dependence (Table 1).
Table 1.
Sample characteristics (N = 262).
| Full sample (n = 262) | Dual mode respondersa (n = 88) | Non-dual mode responders (n = 174) | Significance | |
|---|---|---|---|---|
| Mean age | 35.8 (SD = 10.7) | 35.5 (SD = 10.6) | 36.0 (SD = 10.8) | t(260) = 0.4 |
| Gender female (%) | 180 (68.7%) | 68 (77.3%) | 112 (64.4%) | Chi2(1) = 4.5* |
| Race/ethnicity white (%) | 210 (80.2%) | 72 (81.8%) | 138 (79.3%) | Chi2(1) = 0.2 |
| Education (%) | Chi2(2) = 7.7* | |||
| High school or lower | 43 (16.4%) | 7 (8.0%) | 36 (20.7%) | |
| Some college or trade school | 146 (55.7%) | 51 (58.0%) | 95 (54.6%) | |
| College degree or higher | 73 (27.9%) | 30 (34.1%) | 43 (24.7%) | |
| Presence of other smokers in household (%) | 121 (46.2%) | 41 (46.6%) | 80 (46.0%) | Chi2(1) = 0.0 |
| Mean # of cigarettes/per day (M, SD) | 17.7 (SD = 8.1) | 15.5 (SD = 7.7) | 18.8 (SD = 8.1) | t(260) = 3.2** |
| Number of past quit attempts (M, SD) | 5.3 (SD = 7.1) | 6.0 (SD = 7.2) | 5.0 (7.0) | t(260) = 1.1 |
| Mean baseline nicotine dependence (FTND)⁎ | 5.4 (SD = 2.3) | 4.7 (SD = 2.3) | 5.7 (SD = 2.2) | t(260) = 3.6*** |
| Mean texts sent or received/day | 25.1 (SD = 46.4) | 23.8 (SD = 33.4) | 26.1 (SD = 54.2) | t(189) = 0.4b |
Dual mode responders indicates participants who responded to smoking status indicator on both external survey and internal program assessments within 14 days pre/post the external survey date at least once during follow-up.
Adjusted for unequal variances.
p < .05.
p < .01.
p < .001.
3.2. Research Question 1: consistency of 7-day point prevalence abstinence across modalities
Table 2 shows past 7-day point prevalence abstinence among dual mode responders and consistency of reports. At 1 month, 49 participants (18.7%) did not complete the external survey and 202 participants (77.1%) did not complete the internal program assessments within a time frame of 14 days pre/post the external survey date. A total of 45 participants (17.0%) completed their 1-month external survey and responded to the internal program assessment within 14 days pre/post the external survey date. Of those 45 cases, 88.9% were in agreement and no statistically significant difference in proportion of participants abstinent between modes was detected using McNemar’s test. To correct for agreement expected by chance alone, a Kappa statistic was calculated (κ = 0.24), indicating fair agreement between modes. This low Kappa may reflect the small number of participants indicating smoking in that sample (n = 2; Appendix 1, Table 5), which resulted in a highly imprecise estimate, as shown by the wide 95% confidence interval of −0.21 to 0.68.
Table 2.
Past 7-day point prevalence abstinence among dual-responders (responders to both external survey and internal program assessments within a time frame of 14 days pre/post the web survey date).
| Time | External survey | Internal program assessment | Consistent across-mode reporting | McNemar’s Chi2(1) | Kappa measure of agreement (95% CI) |
|---|---|---|---|---|---|
| 1 month (n = 45) | 95.6% | 88.9% | 88.9% | 1.8 | 0.24 (−0.21, 0.68) |
| 3 months (n = 50) | 76.0% | 84.0% | 88.0% | 2.7 | 0.63 (0.36, 0.89) |
| 6 months (n = 42) | 78.6% | 76.2% | 88.1% | 0.2 | 0.66 (0.39, 0.93) |
Note: CI – confidence interval.
At 3 months, 52 participants (19.9%) did not complete the external survey and 210 participants (80.2%) did not complete the internal program assessments within a time frame of 14 days pre/post the external survey date. A total of 50 participants (19.1%) completed their 3-month external survey and responded to the internal program assessment within 14 days pre/post the external survey date. Of those 50 cases, 88.0% were in agreement and McNemar’s test showed no statistically significant difference between modes. A Kappa of κ = 0.63 indicated substantial agreement between the different modes.
At 6 months, 70 participants (26.7%) did not complete the external survey and 219 participants (83.6%) did not complete the internal program assessments within 14 days pre/post the external survey date. A total of 42 participants (16.0%) completed their 6-month external survey and responded to the internal program assessment within 14 days pre/post the external survey date. Of those 42 cases, 88.1% were in agreement with no significant difference between modes on McNemar’s test. A Kappa of κ = 0.66 indicated substantial agreement between modes. Across 1, 3, and 6 months, dual mode responders reported high rates of abstinence (Appendix 1, Table 5). For example, at 1 month, 39 out of 45 participants (86.7%) reported quitting on both modes and 1 out of 45 (2.2%) reported not quitting on both modes.
3.3. Research Question 2: comparing reporting no smoking among entire sample across modalities
Using the entire sample of 262 participants and assuming missing = smoking for both modes, abstinence rates reported to external survey and internal program assessments within 14 days pre/post the external survey date were compared (Table 3).
Table 3.
Past 7-day point prevalence abstinence across modalities using the full sample (n = 262; external survey and internal program assessments within a time frame of 14 days pre/post the web survey date; using missing = smoking).
| Time | External survey | Internal program assessment | Consistent across-mode reporting | McNemar’s Chi2(1) | Kappa measure of agreement (95% CI) |
|---|---|---|---|---|---|
| 1 month | 30.5% | 16.4% | 82.8% | 30.4* | 0.53 (0.42, 0.65) |
| 3 months | 33.2% | 16.0% | 79.0% | 36.8* | 0.46 (0.34, 0.57) |
| 6 months | 31.7% | 12.2% | 79.0% | 47.3* | 0.42 (0.31, 0.53) |
Note: CI – confidence interval.
p < .001.
At 1-month, 30.5% of participants reported 7-day point prevalence abstinence s to the external survey and 16.4% also reported not smoking to internal program assessments within the 14-day pre/post timeframe. A significant difference was found between these rates and κ = 0.53 indicated moderate agreement between modes.
At 3-months, 33.2% of participants reported 7-day point prevalence abstinence to the external survey and 16.0% also reported not smoking to internal program assessments within the 14-day pre/post timeframe. Rates differed significantly and κ = 0.46 indicated again moderate agreement between modes.
At 6-months, 31.7% of participants reported 7-day point prevalence abstinence to the external survey and 12.2% also reported not smoking to internal program assessments within the 14-day pre/post timeframe. A significant difference was found between rates and κ = 0.42 indicated moderate agreement between modes.
A total of 15.7% of participants were biochemically confirmed abstinent at 6 months (Table 4). Verified rates differed significantly from those reported in external surveys, but did not differ from those obtained by text message assessments. Kappa values indicated moderate agreement for external surveys and fair agreement for text message assessments.
Table 4.
Abstinence across modalities using the full sample, self-reported past 7-day point prevalence abstinence for external survey and internal assessment, saliva cotinine levels of ≤ 15 ng/mL for biochemical verification (n = 262; external survey and internal program assessments within a time frame of 14 days pre/post the web survey date; using missing = smoking).
| Time | Biochemically verified | External Survey | Internal program assessment | Consistent across-mode reporting | McNemar’s Chi2(1) | Kappa measure of agreement (95% CI) |
|---|---|---|---|---|---|---|
| 6 months | 15.7% | 31.7% | 12.2% | Bio-ext: 82.4% | Bio-ext: 38.4* | Bio-ext: 0.53 (0.42, 0.64) |
| Bio-int: 85.1% | Bio-int: 2.1 | Bio-int: 0.38 (0.22, 0.54) |
Note: Bio - Biochemically verified; Ext – External survey; Int – Internal program assessment; CI – Confidence Interval;
p < .001.
4. Discussion
The study examined if internal program data on smoking status, reported via text message, can be a reliable surrogate source for external survey data within the Text2Quit program (Abroms et al., 2014). Results indicate that participants responding to both modes reported smoking status and abstinence rates across modes with good consistency. However, dual mode responders reported high rates of abstinence with few admitting to smoking. This implies that relying exclusively on complete cases from internal program assessments will vastly over-estimate cessation rates.
Using an analytical approach, where missing data on either assessment mode were assumed to indicate smoking, there were significant differences in how individuals reported smoking status, with internal program assessments estimating a lower point-prevalence abstinence at every timepoint. At the 6-month follow-up, abstinence rates obtained via internal data were closer to those biochemically verified compared to external surveys. Notably, abstinence rates based on internal program assessments (12.2%) or biochemical verification (15.7%) at 6-month follow-up are comparable to those in the most recent Cochrane review of mobile phone interventions (Whittaker et al., 2016). These results further encourage testing the use of internal program data as a surrogate for external survey data on smoking cessation outcomes when missing data are assumed to indicate smoking.
Despite significant differences between survey modes in missing = smoking analyses, uncorrected percent agreement ranged between 79% and 82% and Cohen’s Kappa indicated moderate agreement. These findings are in line with and expand the existing literature. Previous studies in tobacco research have compared the agreement between baseline assessments conducted online or via telephone/mail, and reported generally high consistency of findings across survey modes (Callas, Solomon, Hughes, & Livingston, 2010; Graham et al., 2006; Nagelhout et al., 2010). In contrast to these previous studies we used text message assessments and compared follow-up instead of baseline assessments.
More participants completed at least one external survey (88.6%) compared to the internal program assessments (49.6%). Moreover, higher rates of abstinence were reported to external surveys than internal program assessments at 1, 3 and 6-month follow up in the missing = smoking analyses. This is likely due to the rigorous follow-up procedures in place for study evaluation in the Text2Quit trial, with multiple contact attempts, and financial incentives for external survey completion (Abroms et al., 2014; Brueton et al., 2013). The external surveys were used as the primary mode of study data collection, whereas internal program assessments of past 7-day abstinence were not designed for the purpose of study evaluation, and were selected for this analysis only if completed within 14 days pre/post the external survey date. The higher general response rate to external surveys can potentially be attributed to these procedures. To increase response rates of internal program assessments, future studies may adopt similar tactics as those used for external surveys, including multiple reminders (e.g. a text reminder to complete), and where possible, incentives for internal program assessment completion. However, it should also be noted that fewer than 50% of external survey self-reports of abstinence were confirmed by biochemical verification. Compared to biochemical verification results, both text message assessments and external surveys had similar across-mode agreement, though internal program data abstinence rates came closer to those obtained through biochemical verification.
4.1. Limitations
We relied on self-reported smoking status data, which may introduce social desirability bias. Dual-mode responders were more likely to be female, better educated, smoking fewer cigarettes per day, and less addicted to nicotine compared with participants who did not report smoking status to both modes. Education level and level of addiction are positively associated with quitting smoking (Hymowitz et al., 1997), and may introduce a selection bias among dual-mode responders and limit generalizability of results. Predictors of dual-mode non-response identified in the current study are partially aligned with previous research reporting that more socioeconomically deprived pregnant smokers were less likely to respond to text-message assessments (Naughton et al., 2016) and higher text-message intervention dropout among daily compared to non-daily smokers (Cole-Lewis et al., 2016). On the other hand, Cole-Lewis et al. (2016) also reported that women were more likely to drop out, compared to men, which is contrary to our findings. To test the potential impact of selective non-response, we conducted sensitivity analyses for missing = smoking models stratified for significant baseline differences, which remained inconclusive for education and smoking behavior. However, there was a consistent pattern of higher agreement between assessment modes for women compared to men (1 month, women 83%, men 82%; 3 months: women 81%, men 76%; 6 months: women 83%, men 71%). Another important limitation is the timing of internal program assessments of past 7-day smoking did not perfectly sync with the timing of when participants completed external study surveys. We observed relatively low internal program assessment completion rates (only 49.6% of participants completed any internal program assessments), which is in line with previous literature on routine text-message smoking cessation intervention delivery reporting rates of 4%–34% (Cole-Lewis et al., 2016). On the other hand, clinical trials have reported survey completion rates between 74% and 62% (Naughton et al., 2016). When comparing response rates of the current study with previous literature, one should keep in mind that in the Text2Quit trial, participants received program feedback tailored to their responses to internal program assessment. This could have impacted participant response behavior if, for example, only a subset of participants may feel comfortable to share a relapse in these surveys (e.g., help-seekers) and want to engage with program follow-up messages. Future studies using internal program data for evaluation purposes may benefit from separating pure assessment-type text messages from other text messages that aim at customizing intervention components based on participant responses. Lastly, non-response in a text messaging intervention can happen for multiple reasons (e.g., deliberate non-response, change in phone number, loss of phone, relapse to smoking and embarrassment, sustained abstinence and no need in continued intervention, program opt-out) (Cole-Lewis et al., 2016) and the impact of these different reasons on intervention outcome assessment is unclear. A strength of this study is that it is the first to compare both external survey data and internal program data for a text message smoking cessation program.
4.2. Conclusions
The ability to rely on outcome data collected from within a text messaging or other mHealth program has practical advantages. Information collected by text message could alert program managers or researchers of needed program changes that otherwise would not be discovered until the program has ended. While the programming and administration of additional external surveys may require additional resources, program text message assessments can be easily incorporated into interventions and thus save costs. Also, by using internal program data, multiple intervention versions can be evaluated more rapidly (e.g. A/B testing), as well as versions that capture individual variation and changes over time (Hekler et al., 2016). Such rapid evaluation of different program types or components is in line with calls for Agile Science (Hekler et al., 2016), could help in evaluating factorial designs (McClure et al., 2014), and eventually improve mHealth smoking cessation interventions by helping to identify the active/effective ingredients of these programs (Baker, 2017).
The reliability of text message data as a surrogate for external survey data should be further tested in future research, including studies focusing on other addictive substances and health behavior change interventions. Programs should collect external evaluation data as well as develop and test strategies to improve participant compliance with internal program assessments. For example, programs could externally cross-validate a subset of collected data with the aim of attaining high response rates, and subsequently relying on internal program data for timely program monitoring and optimization.
If external data are not available to program evaluators, our findings indicate that participants who respond to internal surveys are more likely to report abstinence than those who do not respond. We recommend using a missing = smoking analysis of internal data as an approximation of the gold standard, biochemically verified quitting, rather than using data of complete case participants who report their quit status internally.
HIGHLIGHTS.
Text messaging interventions have shown promise in helping people quit smoking.
We examined the consistency of self-reported smoking on two assessment modes.
We compared self-reported assessments and biochemical verification results.
Results provide initial support for using text message assessments.
Acknowledgments
Funding
Research reported in this publication was supported in part by grant number 5K07 CA124579-02 to Dr. Abroms, from the National Cancer Institute of the NIH. Support also came from internal funding from the Department of Prevention and Community Health at the George Washington University Milken Institute School of Public Health. Preparation of this manuscript was, in part, supported by the California Tobacco-Related Disease Research Program (TRDRP) grant number 25FT-0009 to Dr. Thrul.
Appendix 1
Table 5.
Cross-tables of past 7-day point prevalence abstinence among dual-responders (responders to both external survey and internal program assessments within a time frame of 14 days pre/post the web survey date).
| Internal program assessment | |||
|---|---|---|---|
| 1 month (N = 45) | |||
| Abstinent | Yes | No | |
| External survey | Yes | 39 | 4 |
| No | 1 | 1 | |
| 3 months (N = 50) | |||
| Abstinent | Yes | No | |
| External survey | Yes | 37 | 1 |
| No | 5 | 7 | |
| 6 months (N = 42) | |||
| Abstinent | Yes | No | |
| External survey | Yes | 30 | 3 |
| No | 2 | 7 |
Table 6.
Cross-tables of past 7-day point prevalence abstinence across modalities using the full sample (n = 262; external survey and internal program assessments within a time frame of 14 days pre/post the web survey date; using missing = smoking).
| Internal program assessment | |||
|---|---|---|---|
| 1 month | |||
| Abstinent | Yes | No | |
| External survey | Yes | 39 | 41 |
| No | 4 | 178 | |
| 3 months | |||
| Abstinent | Yes | No | |
| External survey | Yes | 37 | 50 |
| No | 5 | 170 | |
| 6 months | |||
| Abstinent | Yes | No | |
| External survey | Yes | 30 | 53 |
| No | 2 | 177 |
Table 7.
Cross-tables of biochemically verified and past 7-day point prevalence abstinence rates on both self-report modalities using the full sample at 6-month follow-up (n = 262; external survey and internal program assessments within a time frame of 14 days pre/post the web survey date; using missing = smoking).
| Biochemically verified | |||
|---|---|---|---|
| 6 months | |||
| Abstinent | Yes | No | |
| External survey | Yes | 39 | 44 |
| No | 2 | 177 | |
| 6 months | |||
| Abstinent | Yes | No | |
| Internal program assessment | Yes | 17 | 15 |
| No | 24 | 206 |
Footnotes
Conflict of interest disclosures
The George Washington University/Dr. Abroms has licensed Text2Quit to Wellpass, Inc.; Dr. Lorien Abroms has stock in Wellpass Inc. Dr. Johannes Thrul, Dr. Samuel Simmens, and Judith Mendel, have no financial disclosures.
References
- Abroms LC, Boal AL, Simmens SJ, Mendel JA, Windsor RA. A randomized trial of Text2Quit: A text messaging program for smoking cessation. American Journal of Preventive Medicine. 2014;47(3):242–250. doi: 10.1016/j.amepre.2014.04.010. http://dx.doi.org/10.1016/j.amepre.2014.04.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Abroms LC, Carroll P, Boal AL, Mendel J, Carpenter KM. Integrated phone counselling and text messaging services at Quitlines: An acceptability study. Journal of Smoking Cessation. 2016;11(1):5–11. http://dx.doi.org/10.1017/jsc.2014.11. [Google Scholar]
- Baker TB. The 2016 Ferno award address: Three things. Nicotine & Tobacco Research. 2017;19(8):891–900. doi: 10.1093/ntr/ntx039. http://dx.doi.org/10.1093/ntr/ntx039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brueton VC, Tierney J, Stenning S, Harding S, Meredith S, Nazareth I, Rait G. Strategies to improve retention in randomised trials. The Cochrane Database of Systematic Reviews. 2013;12 doi: 10.1002/14651858.MR000032.pub2. http://dx.doi.org/10.1002/14651858.MR000032.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Callas PW, Solomon LJ, Hughes JR, Livingston AE. The influence of response mode on study results: Offering cigarette smokers a choice of postal or online completion of a survey. Journal of Medical Internet Research. 2010;12(4):e46. doi: 10.2196/jmir.1414. http://dx.doi.org/10.2196/jmir.1414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Christofferson DE, Hertzberg JS, Beckham JC, Dennis PA, Hamlett-Berry K. Engagement and abstinence among users of a smoking cessation text message program for veterans. Addictive Behaviors. 2016;62:47–53. doi: 10.1016/j.addbeh.2016.06.016. http://dx.doi.org/10.1016/j.addbeh.2016.06.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cole-Lewis H, Augustson E, Sanders A, Schwarz M, Geng Y, Coa K, Hunt Y. Analysing user-reported data for enhancement of SmokefreeTXT: A national text message smoking cessation intervention. Tobacco Control. 2016 doi: 10.1136/tobaccocontrol-2016-052945. http://dx.doi.org/10.1136/tobaccocontrol-2016-052945. [DOI] [PubMed]
- Free C, Knight R, Robertson S, Whittaker R, Edwards P, Zhou W, Roberts I. Smoking cessation support delivered via mobile phone text messaging (txt2stop): A single-blind, randomised trial. Lancet. 2011;378(9785):49–55. doi: 10.1016/S0140-6736(11)60701-0. http://dx.doi.org/10.1016/S0140-6736(11)60701-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, Haines A. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: A systematic review. PLoS Medicine. 2013;10(1):e1001362. doi: 10.1371/journal.pmed.1001362. http://dx.doi.org/10.1371/journal.pmed.1001362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Graham AL, Papandonatos GD, Bock BC, Cobb NK, Baskin-Sommers A, Niaura R, Abrams DB. Internet- vs. telephone-administered questionnaires in a randomized trial of smoking cessation. Nicotine & Tobacco Research. 2006;8(Suppl 1):S49–S57. doi: 10.1080/14622200601045367. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hekler EB, Klasnja P, Riley WT, Buman MP, Huberty J, Rivera DE, Martin CA. Agile science: Creating useful products for behavior change in the real world. Translational Behavioral Medicine. 2016;6(2):317–328. doi: 10.1007/s13142-016-0395-7. http://dx.doi.org/10.1007/s13142-016-0395-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hekler EB, Michie S, Pavel M, Rivera DE, Collins LM, Jimison HB, Spruijt-Metz D. Advancing models and theories for digital behavior change interventions. American Journal of Preventive Medicine. 2016;51(5):825–832. doi: 10.1016/j.amepre.2016.06.013. http://dx.doi.org/10.1016/j.amepre.2016.06.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hughes JR, Keely JP, Niaura RS, Ossip-Klein DJ, Richmond RL, Swan GE. Measures of abstinence in clinical trials: Issues and recommendations. Nicotine & Tobacco Research. 2003;5(1):13–25. [PubMed] [Google Scholar]
- Hymowitz N, Cummings KM, Hyland A, Lynn WR, Pechacek TF, Hartwell TD. Predictors of smoking cessation in a cohort of adult smokers followed for five years. Tobacco Control. 1997;6(Suppl 2):S57. doi: 10.1136/tc.6.suppl_2.s57. http://dx.doi.org/10.1136/tc.6.suppl_2.S57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- International Telecommunication Union (ITU) Be he@lthy, be mobile. 2017 Retrieved July 13, 2017, from http://www.itu.int:80/en/ITU-D/ICT-Applications/eHEALTH/Be_healthy/Pages/Be_Healthy.aspx.
- McClure JB, Peterson D, Derry H, Riggs K, Saint-Johnson J, Nair V, Shortreed SM. Exploring the “active ingredients” of an online smoking intervention: A randomized factorial trial. Nicotine & Tobacco Research. 2014;16(8):1129–1139. doi: 10.1093/ntr/ntu057. http://dx.doi.org/10.1093/ntr/ntu057. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nagelhout GE, Willemsen MC, Thompson ME, Fong GT, van den Putte B, de Vries H. Is web interviewing a good alternative to telephone interviewing? Findings from the International Tobacco Control (ITC) Netherlands survey. BMC Public Health. 2010;10(351) doi: 10.1186/1471-2458-10-351. http://dx.doi.org/10.1186/1471-2458-10-351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Naughton F, Riaz M, Sutton S. Response parameters for SMS text message assessments among pregnant and general smokers participating in SMS cessation trials. Nicotine & Tobacco Research. 2016;18(5):1210–1214. doi: 10.1093/ntr/ntv266. http://dx.doi.org/10.1093/ntr/ntv266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- NCD Alliance. Be healthy, be mobile initiative. 2017 Retrieved July 13, 2017, from https://ncdalliance.org/be-healthy-be-mobile-initiative.
- Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobile interventions: Are our theories up to the task? Translational Behavioral Medicine. 2011;1(1):53–71. doi: 10.1007/s13142-011-0021-7. http://dx.doi.org/10.1007/s13142-011-0021-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ubhi HK, Michie S, Kotz D, Wong WC, West R. A mobile app to aid smoking cessation: Preliminary evaluation of SmokeFree28. Journal of Medical Internet Research. 2015;17(1):e17. doi: 10.2196/jmir.3479. http://dx.doi.org/10.2196/jmir.3479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Viera AJ, Garrett JM. Understanding interobserver agreement: The kappa statistic. Family Medicine. 2005;37(5):360–363. [PubMed] [Google Scholar]
- Whittaker R, McRobbie H, Bullen C, Rodgers A, Gu Y. Mobile phone-based interventions for smoking cessation. The Cochrane Database of Systematic Reviews. 2016;4 doi: 10.1002/14651858.CD006611.pub4. http://dx.doi.org/10.1002/14651858.CD006611.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Whittaker R, Merry S, Dorey E, Maddison R. A development and evaluation process for mHealth interventions: Examples from New Zealand. Journal of Health Communication. 2012;17(Suppl 1):11–21. doi: 10.1080/10810730.2011.649103. http://dx.doi.org/10.1080/10810730.2011.649103. [DOI] [PubMed] [Google Scholar]
- Ybarra ML, Jiang Y, Free C, Abroms LC, Whittaker R. Participant-level meta-analysis of mobile phone-based interventions for smoking cessation across different countries. Preventive Medicine. 2016;89:90–97. doi: 10.1016/j.ypmed.2016.05.002. http://dx.doi.org/10.1016/j.ypmed.2016.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]