Optimising text messaging to improve adherence to web-based smoking cessation treatment: a randomised control trial protocol

Abstract

Introduction

Millions of smokers use the Internet for smoking cessation assistance each year; however, most smokers engage minimally with even the best designed websites. The ubiquity of mobile devices and their effectiveness in promoting adherence in other areas of health behaviour change make them a promising tool to address adherence in Internet smoking cessation interventions. Text messaging is used by most adults, and messages can proactively encourage use of a web-based intervention. Text messaging can also be integrated with an Internet intervention to facilitate the use of core Internet intervention components.

Methods and analysis

We identified four aspects of a text message intervention that may enhance its effectiveness in promoting adherence to a web-based smoking cessation programme: personalisation, integration, dynamic tailoring and message intensity. Phase I will use a two-level full factorial design to test the impact of these four experimental features on adherence to a web-based intervention. The primary outcome is a composite metric of adherence that incorporates general utilisation metrics (eg, logins, page views) and specific feature utilisation shown to predict abstinence. Participants will be N=860 adult smokers who register on an established Internet cessation programme and enrol in its text message programme. Phase II will be a two-arm randomised trial to compare the efficacy of the web-based cessation programme alone and in conjunction with the optimised text messaging intervention on 30-day point prevalence abstinence at 9 months. Phase II participants will be N=600 adult smokers who register to use an established Internet cessation programme and enrol in text messaging. Secondary analyses will explore whether adherence mediates the effect of treatment condition on outcome.

Ethics and dissemination

This protocol was approved by Chesapeake IRB. We will disseminate study results through peer-reviewed manuscripts and conference presentations related to the methods and design, outcomes and exploratory analyses.

Trial registration number

NCT02585206.

Strengths and limitations of this study.

• This study will improve adherence to proven web-based cessation interventions, which is critical to leveraging the potential public health impact of this ‘broad reach’ treatment modality. The potential scientific and public health impact of this study is likely to extend beyond web-based cessation programmes to other health risk behaviours.

• The proposed study is innovative in its use of an optimised text messaging intervention as an adherence strategy.

• Study findings will add to the growing knowledge base about the overall effectiveness of Internet cessation programmes and mechanisms through which their population impact on smoking prevalence can be improved.

• A limitation of this study is that it examines a limited number of factors related to adherence. Although multiple factors influence use of web-based cessation programmes, we cannot examine all factors relevant to adherence, but will measure theory-driven constructs to inform our interpretation of results.

• This study also does not specifically target individuals with certain psychiatric or medical comorbidities known to impact smoking cessation rates. If findings are supported, future studies will further refine text message protocols to focus on subgroups at greater risk of smoking relapse.

• Although text messaging is nearly ubiquitous in 2016 at the start of the trial, we are aware of early reports that the number of text messages sent has declined recently, potentially due to the adoption of instant multimedia messaging applications such as Snapchat and WhatsApp. With the increasing use of smartphones and growing use of instant messaging applications, the use of text messaging may continue to decline. However, the number of text messages sent each year remains in the billions, ensuring that this technology will remain relevant for the foreseeable future. In addition, the lessons learnt from this trial about how best to use text messaging to facilitate intervention engagement will be relevant to smartphone apps.

Introduction

Background and rationale

Tobacco use is the leading cause of preventable death in the USA, causing 480 000 premature deaths among adults and nearly $289 billion in total economic burden each year.¹ Reducing population smoking prevalence can save more lives and money than almost any other preventive intervention. Internet interventions are a promising delivery channel for cessation treatment that have potential for enormous public health impact.² The reach of web-based cessation programmes is unparalleled. Millions of smokers search online for quit smoking information each year,³ and hundreds of thousands register on web-based cessation programmes offered by quitlines in 51 US states and territories and Canada,⁴ by hundreds of employers and health plans throughout the US,⁵ or on publicly available, high-volume web-based cessation programmes around the globe.^6–8 Quit rates in Internet programmes range from 18% to 20% at 1 year^9–11 and greater intensity of use yields higher quit rates.^12–17

However, most smokers engage minimally with even the best designed cessation websites, visiting only one to two times and not using many of the interactive tools or community support that promote abstinence.^{15 18–21} As a result, the full potential of Internet cessation programmes to reduce smoking prevalence and save lives is yet to be realised. Poor adherence has been extensively documented across dozens of Internet studies,^{18 22–35} systematic reviews,^36–40 reviews of systematic reviews⁴¹ and meta-analyses⁴² across a range of health behaviours, and is so pervasive it has been described as a ‘fundamental methodological challenge in the evaluation of eHealth applications’ (ref. ³⁰, p. 2).³⁰ This is not a phenomenon unique to one or two websites or to smoking cessation. Adherence is traditionally defined as ‘the extent to which a person's behaviour corresponds with agreed upon recommendations from health care providers’.⁴³ Since many Internet interventions have no specified prescriptions for use,³⁸ adherence may best be defined as ‘the extent to which individuals experience the content of the intervention’,²⁴ or simply, ‘use of the eHealth intervention over time’.⁴⁴ Adherence is typically measured by utilisation metrics such as number of visits to a website, page views, interactive features used and time on site.^{42 44 45}

Periodic prompts and automated reminders can boost intervention adherence.^{46 47} Several studies have examined the effectiveness of email prompts on website engagement,^{23 25 35 48–50} finding that email contacts generally yield more logins,³⁶ but only among a small proportion of study participants.^{49 50} Other elements that improve adherence to web-based cessation programmes include multiple modes of delivery⁵¹ and individually tailored communications.^{22 52} Together, these studies support the use of (1) frequent automated reminders, (2) supplemental modes of communication and (3) a tailored approach to increase adherence to a web-based cessation programme. These converging lines of evidence inform the intervention design of this protocol.

Given the reach of mobile phones, text messaging is an ideal form of supplemental communication for prompts and reminders to promote adherence.^{35 53 54} In October 2014, 90% of US adults owned a mobile phone.⁵⁵ The vast majority of mobile phone owners (81%) use text messaging, including those most likely to smoke: 85% of African Americans, 78% with household income <$30 000/year and 77% with a high school degree send and receive text messages.⁵⁶ Mobile phone owners over 18 years of age send and receive an average of 42 texts/day.⁵⁷ Text message interventions have been shown to increase medication adherence^58–60 and appointment attendance,^61–66 and to promote smoking cessation in the short term^67–69 and long term.⁷⁰ However, few studies have included both web and text programmes components,^{21 71 72} and those that have included both modalities offered to them in parallel with little to no integration between the two platforms, potentially missing powerful synergies. To our knowledge, no studies have examined text messaging to promote adherence to web-based cessation treatment or potential mechanisms of effectiveness.⁷⁰

The mechanisms through which text messages influence behaviour are understudied, and no studies have systematically varied characteristics of text message programmes.⁷³ Based on communication and behaviour change theories, the empirical literature, and our prior work, we identified four aspects of a text message intervention that may enhance its effectiveness in promoting adherence to a web-based cessation programme: personalisation, integration, dynamic tailoring and message intensity.

Personalisation

According to the Elaboration Likelihood Model,⁷⁴ people are more likely to actively process information if they perceive it to be personally relevant. Personally relevant messages may stimulate more thorough consideration of a proposed behaviour change.⁷⁵ Personalisation uses person-specific elements, such as gender, age, name, etc, to enhance the perceived relevance of a message.⁷⁶ Prior studies show that personalisation can increase smokers' attention to written information and the perceived quality of that information.^{77 78} Across a range of health behaviours, personalised text message and web-based interventions have been found to be more efficacious than generic interventions.^{42 79} Personalisation is also important to and desired by text users.^{80 81}

Integration

A meta-analysis by Webb et al⁵¹ found that the effectiveness of Internet interventions for a variety of health behaviours was enhanced by text messages (d=0.81, k=4). However, in the four studies reviewed, the Internet interventions were used simply to gather data needed to tailor the text message programme; the Internet and text programmes operated in parallel with little to no integration. Similarly, a more recent study by Borland et al²¹ tested the combined effects of web and text for smoking cessation, but there was little integration between the two. To date, no studies have examined the effectiveness of a truly integrated web and text intervention in improving treatment adherence and cessation outcomes.^{72 82–84} Addressing these questions has great practical relevance given the number of existing web-based cessation programmes that currently offer text messaging as an adjunct service.⁴ The Webb et al⁵¹ meta-analysis provides an empirical basis for the current study, in which a fully-integrated, multimodal intervention using evidence-based components of a web-based intervention is facilitated via interactive text messages. The goal is to enable users to engage with the components of a web-based intervention via interactive text messages. The ability to interact with and use the tools of a web-based programme via interactive text messages may be more effective in promoting treatment adherence than delivering static text messages that simply refer to a web-based programme.

Dynamic tailoring

Whereas personalisation targets more superficial and often unchanging elements of a message (ie, name, gender, age), tailored communications target theory-driven constructs related to a specific desired outcome.⁸⁵ Decades of research on tailored interventions—including tailored text message interventions⁷⁹—have yielded positive effects on health behaviour change and participation in health promotion programmes.^{79 86 87} Consistent with the Elaboration Likelihood Model,^{74 88} tailored messages are thought to be more effective due to the greater degree of cognitive processing they elicit; tailored messages are more likely to be read, understood, recalled, rated highly and perceived as credible.⁸⁸ However, tailored interventions most often rely on a static assessment of variables used for tailoring. Indeed, most automated text messaging programmes are static in nature, tailored only to baseline variables. Few studies have dynamically tailored communications to deliver ipsative feedback (within-subject change).⁸⁸ With the advent of mobile devices and the ability to gather ‘real-time’ data, there is exciting potential to tailor communications to incorporate changes in an individual's behaviour to provide a ‘smart’ intervention that adapts as the needs of the individual change.⁸⁹ Dynamic text message interventions that change over time in response to a user's interaction with the programme and progress in quitting are a promising target for next generation, scalable systems for behaviour change.⁹⁰ The current study will mimic a face-to-face treatment approach in which participants are given feedback about their treatment progress, reminded about intervention features/content they have not yet used and encouraged to remain engaged with treatment.³⁸ Tailoring text messages based on a participant's previous pattern of engagement with treatment and recommending ‘next steps’ may be more efficacious than requiring users to find their own way through a web-based intervention.

Message intensity

Lastly, text message programmes typically involve an automated programme of messages based around a self-selected quit day.⁶⁷ For example, in the txt2stop study, Free et al⁹¹ delivered an intensive protocol of 35 messages/week for 5 weeks with an abrupt drop to three messages/week for the remaining 26 weeks. A meta-analysis by Head et al⁷⁹ of text message interventions across a range of health behaviours found that intervention efficacy varied by message intensity, with the largest effect size observed for programmes with decreasing intensity (d=0.52, 95% CI 0.44 to 0.61). Decreasing text message protocols tend to taper the intervention from one phase to the next to gradually decrease content delivery. To date, no study has explicitly examined the impact of various levels of message intensity or identified the optimal intensity for a smoking cessation text message intervention. Decreasing message intensity—especially in conjunction with personalisation, interactivity and/or dynamic tailoring—may be more salient and impactful than unchanging intensity.

Findings from this study will yield important insights into improving adherence for web-based cessation programmes around the globe. Many of these programmes offer text messaging as an adjunct service alongside a web-based programme, but none to date that integrate Web and text programmes so that they seamlessly and dynamically work together. Results from this study will identify strategies for integrating these services to promote adherence and improve quit rates, and will identify specific features and functionality to include in a text programme.

The potential scientific and public health impact of this study is likely to extend beyond web-based cessation programmes. Millions of adults use the Internet for assistance with addictions and other health behaviours.⁹² Across healthcare, adherence is a problem that plagues numerous therapies.⁴³ Results from this study may inform advances in intervention design to better engage users and sustain their involvement across a range of evidence-based programmes. Given the demonstrated use of web-based interventions among hundreds of thousands of minimally-engaged smokers who want to quit, it is critical to advance scientific understanding about how to better engage users so they receive the optimal dose of treatment necessary for abstinence. The need to improve adherence is clear: even the best treatments will have little impact if they are not used.

Objectives

The overarching goal of this study is to more effectively engage the hundreds of thousands of minimally engaged smokers already using the Internet to quit smoking. As shown in figure 1, this two-phase study will: (1) identify the factors in a text message intervention that yield optimal adherence to a web-based smoking cessation intervention (Phase I), and (2) examine the comparative effectiveness of a web-based cessation intervention alone (WEB) and in conjunction with the optimal-adherence text messaging intervention (WEB+TXT; phase II).

Figure 1.

Overall research design.

Specifically, in phase I, the study will examine the impact of four experimental features of a text message intervention on a composite metric of adherence to a web-based cessation intervention. We hypothesise that personalisation, integration, dynamic tailoring and decreasing message intensity will have positive effects on adherence. Phase II will address two aims. Aim 1 will examine the comparative effectiveness of a web-based cessation intervention alone (WEB) or in conjunction with an optimised text messaging intervention (WEB+TXT) with regard to 30-day point prevalence abstinence at 9 months postrandomisation (primary outcome) and adherence metrics (secondary outcomes). We hypothesise that WEB+TXT will yield higher rates of abstinence and adherence than WEB. Aim 2 will examine whether the impact of treatment assignment on cessation is mediated by adherence. We hypothesise treatment group differences in 30-day point prevalence abstinence at 9 months will be mediated by greater levels of adherence at 3 months to a web-based cessation programme.

Trial design

This two phase study will be conducted with registered users on BecomeAnEX.org, an established and widely used smoking cessation website run by Truth Initiative. Phase I involves the initial development and optimisation of the text message intervention. We will examine the impact of four experimental text message intervention features on smokers' adherence to a web-based cessation intervention during the first 3 months of programme enrolment. We will utilise a full factorial design where participants will be randomised to 1 of 2 levels of each of the following features: (1) personalisation (yes/no), (2) integration (yes/no), (3) dynamic tailoring (yes/no) and (4) message intensity (standard vs decreasing). The primary outcome in phase I will be a composite metric of website adherence. Phase II involves a two-arm randomised trial that compares WEB alone to WEB plus the text message intervention from phase I that yields optimal adherence (WEB+TXT). The randomised trial will use a repeated-measures design, with assessments at baseline, 3, 9 and 15 months postrandomisation. Follow-ups at 3, 9 and 15 months correspond to 0, 6 and 12 months post-treatment. The primary outcome is 30-day point prevalence abstinence (ppa) at 9 months. Other outcomes include motivation to quit smoking, number of quit attempts and continuous abstinence.

Design considerations

Text messaging versus smartphone app

We chose a text message intervention over a smartphone app for several reasons: (1) a majority of mobile phone owners across most demographic groups use text messaging (excepting those aged 65+ years), especially young adults aged 18–29 years and minority groups such as African Americans and Hispanics;⁹³ (2) 80% of mobile users send/receive text messages compared to only 43% who download apps;⁹³ (3) text messaging is a proven cessation modality whereas smartphone apps are not; (4) app installation may feel intrusive: 57% of app users uninstall/decline to install apps due to privacy concerns.⁹⁴ Text programmes require minimal personal data and may feel less intrusive; (5) smartphone penetration (56%) lags behind high rates of mobile use.⁹⁵

Use of full versus fractional factorial design

Fractional factorial designs for evaluating multicomponent interventions have become popular^{96 97} and have been used in several web-based cessation trials.^98–100 Their popularity is partly based on the ability to rapidly screen a large number of intervention components for main effects, using a fraction of the treatment component combinations required for a full factorial design. A potential downside of fractional factorial designs is that they require a formal refinement phase to resolve interactions aliased with main effects and to evaluate the optimal dose of factors with more than two levels. The addition of a refinement phase makes the total sample size and study duration for phase I hard to estimate in advance, since the exact nature of follow-up experiments to be conducted is highly dependent on the screening phase. Given only N=4 factors and their binary nature—and the fact that we incur no costs for participant recruitment and have an ample pool to recruit from—we will use a full factorial design that makes use of all 2⁴=16 combinations of intervention components. Since we have no aliasing issues to resolve and no need to conduct additional experiments to determine optimal factor doses, we can finalise our text intervention based on phase I results where sample size and duration can be fixed in advance.

Decision to power for interactions versus main effects

As shown by Chakraborty et al,⁹⁷ 2^k factorial designs are grossly underpowered for detecting simple effects (ie, changes in individual factors while keeping all remaining factors fixed). Rather, they gain power by focusing on main effects (ie, averages of all 2^k−1 simple effects obtained by varying an individual factor at fixed levels of the remaining factors). For example, a simple effect of personalisation would correspond to comparisons of arms 1 vs 9 in figure 1, whereas its main effect would correspond to a comparison of arms 1–8 (personalisation=no) with 9–16 (personalisation=yes). A focus on main effects is acceptable during a screening experiment to identify inactive components rather than in evaluating the usefulness of a particular treatment combination compared to the effects of each component alone. Intervention components with null effects will be dropped from further consideration, while optimum levels for the remaining factors will be set based on their signs in the regression model, with levels having positive signs being associated with higher engagement. In principle, one could observe a situation in which the joint effect of two protective factors is lower in magnitude than the sum of their main effects; such subadditivity is not a concern given the negligible cost of offering additional intervention components, as long as the joint effect of the two factors exceeds each of their main effects considered alone. Failure of the latter condition would correspond to a strongly antagonistic interaction that manifests itself in some pharmacological trials, but which we consider unlikely to occur in a web intervention. We have chosen a sample size that allows us to identify and explain any such strongly antagonistic two-way interactions.

Methods and analysis

This clinical trial protocol was prepared in accordance with the Standard Protocol Items Recommendations for Interventional Trials (SPIRIT) checklist.^{101 102}

Phase I Methods

Participants

In phase I, participants will be N=860 adult US smokers who are new registered users on BecomeAnEX (ie, no prior use of the site as determined by IP address and registration data) and who fully enrol in the BecomeAnEX text messaging programme. During registration, participants must indicate current smoking (every day or some days), age 18 years or older, and US zip code as determined by IP address.

Enrolment

New BecomeAnEX members who meet eligibility criteria will be automatically randomised to 1 of 16 arms of the factorial. Study enrolment will be conducted in 10 months. Based on our prior work, we conservatively estimate that N=3/day (∼90/month) new members of BecomeAnEX will enrol in text messaging.

Allocation and blinding

Randomisation will be stratified by whether participants access the Internet on their cell phone (yes/no), since access to BecomeAnEX via mobile site may influence adherence. Randomisation will be automated using a computer algorithm. The allocation sequence for the full factorial will be generated by the study statistician at study start-up and uploaded into the web-based clinical trials management system that will automate its implementation. All investigators and research staff will remain blinded to treatment assignment throughout the conduct of the study.

Phase I Interventions

Participants are free to use the interventions described below for as long as they desire. Proactive emails from the WEB programme can be stopped at any time, and users can unsubscribe from the text message programme at any time. There are no restrictions on use of other cessation interventions during the study period.

Web-based cessation programme

BecomeAnEX is an evidence-based cessation programme that was launched in 2008 by Truth Initiative (formerly the American Legacy Foundation).^{19 103} Based on the Clinical Practice Guideline for Treating Tobacco Dependence¹⁰⁴ and consistent with Social Cognitive Theory,¹⁰⁵ the site educates smokers and provides the tools necessary to enhance self-efficacy for quitting. BecomeAnEX guides and supports smokers through the following interactive components: (1) a Quit Date tool that assists users in selecting a quit date; (2) Cigarette Tracker exercise to identify smoking triggers; (3) Beat Your Smoking Triggers exercise to identify strategies to dissociate cigarettes from triggers; (4) Build Your Support System exercise to identify helpful supporters; (5) Choose a Quit Smoking Aid exercise, in which users indicate their plans for pharmacotherapy use; and (6) Community, a large online network of current and former smokers who communicate via personal messages sent directly between members, public ‘wall posts’ (comments on a user's profile page) and blog posts/replies. The site can be browsed anonymously but to save information, visitors must register. Registration includes smoking status, gender and age, email, username/password and request for email messages. Sign-up for text messaging occurs during registration, and involves providing a mobile number and affirming the request. A mobile web version includes the full functionality of the site.

Text messaging

Truth initiative has developed a fully-automated text messaging programme that is available via BecomeAnEX. Users can set a quit date and receive scheduled messages tailored to their quit date; quit dates may be changed or cleared as frequently as a user desires. Messages tailored to quit date encourage use of evidence-based cessation methods (eg, nicotine replacement therapy, peer support), praise success, inform users about addiction and reinforce benefits of quitting. Users who do not set a quit date receive scheduled messages tailored to their enrolment date. These messages prompt users to set a quit date, use evidence-based resources, and reinforce benefits of quitting and harms of continuing to smoke. Keywords (eg, ‘COPE,’ ‘SLIP,’ ‘MOOD’) allow users to request on-demand support related to cravings, relapse and negative affect. ‘STOP’ terminates the programme at any time. The existing text message intervention will be modified to serve as the base case (arm 1) in phase I. The intervention will be neither personalised, nor integrated with BecomeAnEX, nor dynamically tailored, and will use a standard message intensity delivery schedule (see study arm 1 in figure 1). For study arms 2–16, we will modify approximately half the messages to ensure adequate differentiation between factor levels, while preserving a coherent user experience. We will user-test messages prior to the launch of phase I to obtain feedback about their persuasiveness, relevance, likelihood of stimulating a response and appeal. We will also obtain user feedback about message intensity (eg, desired message frequency, duration of text intervention) and integration between Web and text via interactive messages.

Factor 1—Personalisation

Personalisation has been implemented in varying ways in text message studies, ranging from participant name only¹⁰⁶ to name plus numerous pieces of personal information.¹⁰⁷ We will personalise text messages using the participant's BecomeAnEX username and gender.

Factor 2—Integration

Interactive messages will facilitate engagement via text with the 6 interactive intervention components of the BecomeAnEX website listed above. Set Quit Date is already a feature of the text message programme; the system tailors messages around a quit date. Unlike most other text message programmes that require a quit date—and consistent with our aim to improve adherence to a web-based intervention among all users—we will deliver 3 months of text messages regardless of whether a user sets a quit date. When a user sets a quit date via the website, it will trigger text messages tailored to that quit date. All participants, regardless of randomisation assignment, will be able to set their quit date via text message and have this information reflected on the website. To enable participants to interact with the Cigarette Tracker, Beat Your Smoking Triggers, Build Your Support System and Choose a Quit Smoking Aid exercises via text message, an initial text message will query the user for a response. Subsequent messages will elicit additional user input (eg, other triggers), highlight relevant content on BecomeAnEX (eg, how to cope with boredom) and reinforce use of the site. To facilitate interaction in the Community via text message, we will develop a new Community feature called ‘QuickTips’ that will enable BecomeAnEX members to submit text message-sized tips to help support other members. We will enable BecomeAnEX members to submit QuickTips via website, mobile site, or text message. Submissions will undergo review/approval by the BecomeAnEX Administrator before being distributed via text message. The Administrator will augment the QuickTips library as needed, excerpting community content to craft ‘user-generated’ tips. Text users can also request QuickTips for on-demand ‘peer support’.

Factor 3—Dynamic tailoring

The general principle guiding implementation of this feature is that messages will be individually tailored to remind/reinforce users about BecomeAnEX information/tools they have already used, or to prompt users to take actions they have not yet taken. Although BecomeAnEX and the text message system are separate systems, they will communicate via an application programming interface, which will allow BecomeAnEX to alert the text message system of site utilisation. Real-time site utilisation data will tailor text messages to encourage the participant to use components of the website they have not yet used or reinforce ongoing use.

Factor 4—Message intensity

The text message intervention will be 12 weeks in duration. For the standard intensity protocol, participants will receive a consistent number of messages each day. The decreasing intensity protocol will deliver approximately the same total number and content of messages as the standard intensity protocol, but with additional tapering steps designed to mirror the declines in cravings and withdrawal typically experienced by a smoker after quitting.¹⁰⁸ The decreasing protocol will gradually reduce the number of daily messages, rather than sharply decrease as in the standard intensity protocol. We will examine the relationship between ‘dose’ of text messages received and our composite adherence metric.

Phase I Measures

Phase I will rely on data from 4 sources: (1) BecomeAnEX registration data, (2) automated tracking data gathered through Adobe Analytics¹⁰⁹ software, (3) automated tracking data stored in unified event logs and (4) text message utilisation data. Use of the following components of BecomeAnEX will be extracted from unified event logs: Set Quit Date, Cigarette Tracker, Beat Your Smoking Triggers, Choose your Quit Smoking Aid, Build Your Support System and Community (eg, # of wall posts made/received, blog posts/replies, messages sent/received, QuickTips submitted). From our text message system, we will extract replies to interactive text messages including engagement with the six interactive features of BecomeAnEX, unsubscribe rates, modal day of unsubscribe, number of days enrolled, messages received and keyword requests. All text interactions are date/time stamped.

Primary outcome

The primary outcome of phase I will be a composite metric of adherence for each participant created using a weighted sum of general measures of engagement (eg, visits, page views, time on site) and specific feature utilisation shown to predict abstinence during 3 months postenrolment. The weights for these utilisation metrics will be given by the regression coefficients of a logistic regression model that we have already developed to measure the effects of website engagement on three-month abstinence rates in the control arm of a prior ongoing web-based cessation study.¹¹⁰ The resulting adherence metric has the advantage that it is continuously distributed, even if some of the original utilisation metrics are binary or count data. Therefore, it can be analysed as the primary outcome for phase I using standard linear regression techniques, possibly after a normalising transformation. Treatment duration of 3 months will provide sufficient time to examine the impact of the text message intervention on adherence, since most non-usage attrition happens within the first 3 months.¹⁹

Phase I Data analysis plan and sample size calculations

Analysis plan

We expect the primary outcome to be continuously distributed, although it may require a symmetrising transformation to reduce skewness. Given our prior experience with these data, we expect to find a few outliers that represent heavy website users. Rather than discard such valid data points, we will reduce their impact by winsorisation (similar to trimming).¹¹¹ Once we are satisfied about the adequacy of the normal approximation, we will analyse the data from our factorial designs using the approach detailed in the paper by Chakraborty et al.⁹⁷

Sample size calculations

To be conservative, we used a sample size for phase I that allows us to detect small main effects of any of the four factors of interest (d=0.25) or moderate two-way interactions (d=0.50) with power 80% at a two-sided significance level of α=0.05/10 (multiplicity adjustment based on four main effects and six two-way interactions in the model). Effect sizes for interactions were chosen to be such that they could nullify the presumed beneficial main effects of the factors involved. The sample size required is N=430 per factor level, or N=860 overall. These calculations were not adjusted for attrition since we will have adherence data on all participants.

Phase II Methods

Subjects

We will recruit a separate sample of N=600 adult smokers who register on BecomeAnEX and sign up for text messaging during registration. To be invited, participants must be adult current smokers (every day/some days) who register on BecomeAnEX and enrol in the text message programme. Invited participants will complete a separate screening process to confirm eligibility. To maximise generalisability, we have no inclusion criteria related to motivation for cessation.

Recruitment and enrolment

Phase II participants will not be enrolled automatically as in phase I, but must respond to a study invitation and complete online eligibility screening, provide informed consent, undergo a baseline assessment and fully enrol in the text messaging programme. Study enrolment occurs online via our web-based clinical trials management system. Randomisation will only occur after a potential study participant has completed the online enrolment process (completed baseline survey) and replied ‘OK’ to the welcome text message to confirm text message enrolment. Randomisation will be stratified by gender, age (≤30, 30+ years), and whether participants access the Internet on their cell phone (yes/no) since age and access to BecomeAnEX via mobile site may influence adherence and gender may influence cessation outcomes. A computer algorithm will automate random allocation. Recruitment will be conducted over 14 months.

Interventions

WEB: Participants will have full access to BecomeAnEX. They will not receive any intervention via text message. Informed consent will explain the possibility of being randomised to a non-text treatment arm. WEB+TXT: Participants will have full access to BecomeAnEX and the optimal-adherence text intervention developed in phase I. Participants are free to use the interventions for as long as they desire. Proactive emails from the WEB programme can be stopped at any time, and users can unsubscribe from the text message programme at any time. There are no restrictions on use of other cessation interventions during the study period. Use of other quit methods will be assessed at all follow-up intervals.

Retention

We expect at least 70% follow-up at 9 months. To maximise follow-up we will: (1) provide clear information about the study at the outset, including expectations for follow-up; (2) reimburse participants $50 per follow-up; and (3) emphasise the importance of survey completion regardless of smoking status. If follow-up rates are lower than expected early in the trial, we will consider shortening the 9-month follow-up to gather only abstinence outcomes.

Phase II Measures

Assessments will occur at baseline, 3, 9 and 15 months postrandomisation. The baseline survey will be conducted online and hosted on a secure server. Mixed-mode follow-up (email, phone, text) will be employed. Telephone surveys will be conducted by research staff blind to treatment. Text messages have demonstrated moderately high reliability (k=.66) compared with web-based surveys in assessing smoking outcomes¹¹² and will be used as a final means of gathering abstinence data from non-responders. Most measures listed below are standard instruments used in cessation studies, and are reliable when administered via the Internet.^{113 114}

Baseline variables

To characterise the sample and examine moderators, we will gather information on: demographics (age, sex, marital status, race, ethnicity, employment and education); current smoking behaviour and smoking history (smoking frequency and rate; quitting history, including quit methods; current and past use of other tobacco products); nicotine dependence assessed by the Heaviness of Smoking Index;¹¹⁵ motivation to quit smoking measured with the Readiness Ladder;¹¹⁶ mobile phone type/use (average number of text messages sent/received each day, cell phone use to access the Internet, send or receive email,⁹³ data plan on phone¹¹⁷ and where, how and how often they access the Internet); and smoking cessation self-efficacy measured with the short form of the Smoking Situations Confidence Questionnaire.¹¹⁸

Outcome measures

The primary outcome is self-reported 30-day ppa at 9 months postrandomisation but we will gather abstinence data at all follow-ups. Other smoking-related outcomes will include change in motivation to quit, quit attempts, 7-day ppa and continuous abstinence measured at each follow-up. Intervention satisfaction in both conditions will be measured with items about overall satisfaction, perceived helpfulness, whether the intervention met their expectations (1=not at all, 5=very much) and whether they would recommend the intervention to a friend (1=definitely not, 5=definitely would). Satisfaction with frequency/duration of text messages, and perceptions about Internet and text message integration, will also be measured.⁷¹ To assess perceived message relevance, participants will be asked whether text messages ‘were written personally for you’²² and ‘were directed at you personally’ (1=not at all, 7=very much).⁷⁶

Mediating variables

We will examine the same website utilisation metrics as in phase I from Adobe Analytics¹⁰⁹ and unified event logs. Adobe Analytics will provide metrics for contact time (total time spent logged into the website), number of website sessions (number of return visits to the website), use of the static content on the site (number of page views) and number of videos watched. Unified event logs will provide data on use of the six interactive components of BecomeAnEX and the platform on which they were used (ie, website or mobile site). We will extract replies to interactive text messages including engagement with the six interactive features of BecomeAnEX and number of keyword requests from the text message system. Unsubscribe rates, modal day of unsubscribe, number of days enrolled and messages received, will be extracted. All text message interactions are date and time stamped.

Phase II data analysis plan and sample size calculations

The distributional properties of continuously scaled variables will be examined to determine the need for normalising transformations. Next, we will determine whether the groups show large standardised mean differences at pretreatment on demographic characteristics, psychosocial variables or smoking variables. Although the large sample size should preclude finite sample randomisation imbalances, should such between-group differences be found, we will correct for them via regression adjustment.

Outcome analyses

Our primary outcome for aim 1 is self-reported 30-day ppa. Differences in abstinence rates between the two treatment conditions will be evaluated at our primary endpoint of 9 months postrandomisation, as well as the secondary endpoints of 3 and 15 months post-randomisation. To account for within-subject correlation due to the repeated-measures aspect of our study, we will employ the Generalised estimating equation, which extends generalised linear model methodology to correlated data in PROC GENMOD of SAS/STAT (SAS Inc, Cary, North Carolina, USA). Analyses will be conducted first using an intention-to-treat (ITT) principle, analysing data from all participants randomised to treatment and counting as smokers those lost to follow-up (missing=smoking).

Moderator analyses

We will examine potential moderators of the intervention-smoking cessation relationship (eg, gender, baseline stage of motivational readiness, nicotine dependence). Effect modification will be conducted by analysing interactions between treatment and selected variables.

Mediator analyses

Primary aim 2 hypothesises that adherence mediates the intervention-cessation relationship. We will establish mediation using the MacKinnon approach.¹¹⁹ As explained in Cerin and MacKinnon¹²⁰ and implemented by Papandonatos et al,¹²¹ behavioural researchers ought to determine whether: (A) the intervention successfully acted on the putative mediator (ie, ‘Action Theory test’); (B) changes in the mediator were indeed predictive of changes in the target behaviour suggested by the conceptual framework underpinning the intervention over and above any direct treatment effects (ie, ‘Conceptual Theory test’); and (C) these conditions held simultaneously for each mediator of interest, indicating that the corresponding mediational pathway accounted for at least part of the relationship between the intervention and the target behaviour (ie, ‘Mediation test’).

Missing data

We expect less than 25% missing data at any time. If a participant refuses follow-up, we will censor the data at the point of loss of contact. Under an ITT approach, participants who had been considered non-smokers up to the point of loss will be considered smokers at future data points. One concern is that this approach is sensitive to differential attrition across study arms and tends to overestimate precision of estimates of treatment effects.¹²² Therefore, we will supplement ITT analyses with a multiple imputation procedure that assumes the odds of missingness vary for smokers and non-smokers lost to follow-up. This model falls under the missing-not-at-random (MNAR) characterisation of missingness mechanisms by Little and Rubin,¹²³ and requires knowledge of the OR relating missingness to smoking. Since this OR is in practice unknown, we will conduct a sensitivity analysis to assess its impact on the estimate and significance of the intervention effect.^{124 125}

Sample size

Efficacy estimates for WEB are based primarily on the BecomeAnEX trial¹¹⁰ and the BecomeAnEX Outcome Evaluation,¹⁹ augmented by results from our iQUITT Study¹²⁶ and the American Cancer Society (ACS) trial.^{12 127} In our ongoing BecomeAnEX trial,¹¹⁰ 30-day ppa using ITT at 9 months in the Internet-alone arm is 8.8%. In the BecomeAnEX Outcome Evaluation,¹⁹ 30-day ppa using ITT at 6 months was 9.9%. Quit rates for the web programmes in The iQUITT Study ranged from 12.2%-14.4% at 6 months (30-day ppa ITT). In the ACS trial¹² 7-day ppa (ITT) at 13 months was 8–12% across all five interactive websites and 10% for the static site. We conservatively estimate that the 30-day ppa rate at 9 months for WEB will be 9% under ITT. Efficacy estimates for WEB+TXT are based on several converging lines of evidence: (1) a meta-analysis by Webb et al⁵¹ found that the parallel use of text messaging in Internet interventions had large effects on behaviour change (d=0.81, k=4); (2) abstinence rates in the BecomeAnEX Outcome Evaluation¹⁹ at 6 months were roughly doubled among those who used the community (OR=2.22, 95% CI 1.34 to 3.69, p=0.002) and separation exercises (OR=1.91, 95% CI 1.00 to 3.65, p=0.05) two or more times compared to those with no utilisation; increased adherence to these features facilitated by text messages is expected to increase abstinence similarly; (3) Brendryen et al⁷² tested a multicomponent cessation intervention that included parallel Internet and text messaging, which yielded 12-month repeated ppa of 20% under ITT. We conservatively estimate that the 30-day ppa rate at 9 months for WEB+TXT will be 16.5% under ITT, corresponding to an intervention OR=2.0, which can be detected with 80% at two-sided α=0.05 using N=300 per study arm (N=600 total).

Study monitoring

At the start of the study, a Data Safety Monitoring Committee will be established, comprised of the Principal Investigator, Data Analyst, Biostatistician, Technical Lead and Project Manager. This committee will discuss protocol development and will review scientific, safety and ethical issues related to the study design, and approve plans for data integrity. The committee will meet every 2 weeks to review the following information in detail: (1) participant accrual rate, (2) participant drop-out and the reasons for drop-out, (3) targeted enrolment status, and (4) major and minor problems related to treatment arm assigned.

The overall risk is judged to be very low for phase I and phase II. Study participants who attempt to quit smoking will likely experience some nicotine withdrawal symptoms that may include anxiety, restlessness, anger, irritability, sadness, problems concentrating, appetite change and weight gain, insomnia and decreased heart rate. There is no reason to believe that participation in this study would worsen nicotine withdrawal symptoms or that symptoms would differ based on randomisation assignment in either phase I or phase II. Exposure to evidence-based information and support for smoking cessation in both the Internet and text message interventions is expected to attenuate withdrawal symptoms associated with smoking cessation that may occur during phase I or phase II. Given the nature of the trial, we do not anticipate the need for interim analyses or stopping guidelines. Serious adverse events are unlikely, but will be reported immediately on discovery by study staff to the Principal Investigator, who will notify the IRB and NIH Project Officer within 24 h.

Participant timeline—phase I and phase II

The main outcome of interest for phase I is a composite metric of adherence for each phase I participant created using a weighted sum of general engagement metrics (eg, website visits, page views, time on site) and specific feature utilisation shown to predict abstinence during 3 months postenrolment. There is neither baseline nor follow-up assessment for participants to complete as adherence metrics are collected automatically. Participants will receive the text message intervention for 12 weeks and participants can discontinue at any time by texting back ‘STOP’. The timeline for phase I is depicted in figure 2.

Figure 2.

Phase I participant timeline.

The main outcome of interest for phase II is 30-day point prevalence abstinence (ppa) at 9 months. Secondary outcomes include 30-day ppa at 3 and 15 months, and adherence metrics. Participants will be assessed at baseline to gather demographics; current smoking behaviour and smoking history, nicotine dependence assessed by the Heaviness of Smoking Index; motivation to quit smoking measured with the Readiness Ladder; mobile phone type and use; and smoking cessation self-efficacy measured with the short form of the Smoking Situations Confidence Questionnaire. The intervention period will past for 3 months postrandomisation. Follow-up surveys will take place for all participants at 3, 9 and 15 months postrandomisation, during which they will be assessed on smoking-related outcomes, including change in motivation to quit, quit attempts, 30-day ppa, 7-day ppa and continuous abstinence; intervention satisfaction; and perceived text message relevance. The baseline survey will be conducted online. Mixed-mode follow-up (email, phone, text) will be employed for all follow-up assessments. The timeline for phase II is depicted in figure 3.

Figure 3.

Phase II participant timeline.

Ethics and dissemination

Research ethics approval

Institutional Review Board approval for the study was provided by Chesapeake Institutional Review Board.

Protocol amendments

Any protocol modifications will be submitted for approval to Chesapeake IRB and reflected (as needed) in trial registry information available on ClinicalTrials.gov.

Consent

New registrants on BecomeAnEX.org will be automatically enrolled in phase I following BecomeAnEX registration if they meet the eligibility criteria and complete BecomeAnEX text message enrolment by replying ‘OK’ to the welcome message. To register on BecomeAnEX, individuals must agree to the site's Terms of Use and Privacy Policy. The Privacy Policy makes explicit that (1) Truth Initiative automatically collects information about its users and their use of the site, (2) information is used for research and quality improvement purposes only, and (3) personal information is kept confidential. Eligible individuals will be automatically randomised to 1 of 16 arms of the factorial design. Their use of the BecomeAnEX website and text messages will be tracked for 3 months. No additional screening information will be requested or obtained and no separate informed consent will be solicited as the registration process makes it clear that the Truth Initiative may monitor all registered users' use of the website and that they are explicitly agreeing to receive text messages.

In phase II, new registered users on BecomeAnEX are presented the informed consent page and provided detailed information about the study (see online supplementary appendix A). Randomisation will only occur after an eligible study participant has completed the online enrolment process (confirmed eligibility, indicated informed consent, confirmed contact information and completed baseline survey) and replied ‘OK’ to the welcome text message to confirm text message enrolment.

Confidentiality

Confidentiality will be protected at all times and potential risks minimised systematically. During BecomeAnEX registration, participants create a username and password that they use to log into the website. Each use of BecomeAnEX involves a session ID unique to the user, and the date and time of access. BecomeAnEX uses industry standard security protocols. Users are automatically logged out of BecomeAnEX after 30 min of inactivity. The BecomeAnEX Privacy Policy will be in effect for all participants enrolled in the project. Transactional data from the site are loaded on a daily basis into a local data warehouse that is provided with both physical and electronic protection. Confidentiality of data will be maintained by numerically coding all data, by keeping identifying information separate from research data and by keeping all data electronically protected. Identifying information will not be reported.