Understanding attrition from international internet health interventions: a step towards global eHealth

Abstract

Worldwide automated Internet health interventions have the potential to greatly reduce health disparities. High attrition from automated Internet interventions is ubiquitous, and presents a challenge in the evaluation of their effectiveness. Our objective was to evaluate variables hypothesized to be related to attrition, by modeling predictors of attrition in a secondary data analysis of two cohorts of an international, dual language (English and Spanish) Internet smoking cessation intervention. The two cohorts were identical except for the approach to follow-up (FU): one cohort employed only fully automated FU (n= 16 430), while the other cohort also used ‘live’ contact conditional upon initial non-response (n= 1000). Attrition rates were 48.1 and 10.8% for the automated FU and live FU cohorts, respectively. Significant attrition predictors in the automated FU cohort included higher levels of nicotine dependency, lower education, lower quitting confidence and receiving more contact emails. Participants' younger age was the sole predictor of attrition in the live FU cohort. While research on large-scale deployment of Internet interventions is at an early stage, this study demonstrates that differences in attrition from trials on this scale are (i) systematic and predictable and (ii) can largely be eliminated by live FU efforts. In fully automated trials, targeting the predictors we identify may reduce attrition, a necessary precursor to effective behavioral Internet interventions that can be accessed globally.

Internet-administered health interventions are rapidly gaining empirical support. To date, Internet interventions have been shown to help individuals stop smoking (Muñoz et al., 2009; Shahab and McEwen, 2009) and lose weight (Weinstein, 2006), reduce depression and anxiety (Griffiths et al., 2010), encourage effective management of diabetes (McKay et al., 2001), and improve insomnia (Ritterband et al., 2009). Fully automated interventions—those that can be completed without ‘live’ human contact—may be especially useful for highly prevalent and costly global health problems such as smoking, depression and diabetes, among others. Automated evidence-based Internet interventions can provide a cost-effective alternative to traditional interventions, which are consumable (no pill or therapy hour can ever be used again once used by their recipients) and difficult to access in many parts of the world. The scalability of fully automated, evidence-based Internet interventions makes it economically feasible to simultaneously deliver treatment to many individuals, without consuming the treatment resource or diminishing its effectiveness (Muñoz, 2010). For example, smoking is a global health problem that may kill 1 billion people during this century (World Health Organization, 2008). Many of these deaths occur in low- or middle-income countries (World Health Organization, 2011) where effective smoking cessation options are either unavailable or, when available, too difficult or too expensive to access for effective widespread use. Given the prevalence of smoking, traditional interventions are highly unlikely to reach the number of smokers seeking help to quit. For example, of the participants in a recent Internet smoking cessation study only 3.3% had sought cessation help from a doctor and only 1.7% had attended a smoking cessation group. The most used quitting aid was nicotine gum, but even that method had only been used by 12% of smokers (Muñoz et al., 2012). Conversely, an automated smoking cessation Internet intervention could easily reach individuals with limited local cessation options at the users' convenience, and do so free of charge.

Before automated Internet interventions are implemented at a global level, they should be evaluated. Attrition is a primary barrier to evaluating large-scale automated Internet interventions, with levels often reaching 60–80% (Eysenbach, 2005). Attrition presents both a challenge in the effective evaluation of trials and services, and an opportunity: if analyzed systematically, attrition patterns can be used to identify characteristics of participants and of interventions associated with disengagement from automated Internet programs. These data will inform intervention modifications that may increase engagement.

There are two types of attrition from Internet outcome studies: (i) attrition from the intervention itself (lack of site utilization) and (ii) attrition from the follow-up (FU) assessments (Eysenbach, 2005). A trial could have attrition from either or both of these sources. Thus, modifications to reduce attrition could be focused on either or both sources. This study addresses attrition from FUs.

Researchers often choose to manage FU attrition in Internet interventions with cohort maintenance strategies such as live telephone contact and monetary incentives. While these strategies can improve retention, they do not do so consistently (Khadjesari, 2011), and may be prohibitively expensive and logistically challenging when conducting Internet intervention trials with thousands of participants worldwide. Ideally, a self-help automated intervention should be evaluated exactly as it will be implemented. Introducing live FU components or financial incentives exposes participants to powerful motivators that will be absent once the intervention is disseminated, thus reducing the ecological validity of results. Therefore, the most effective long-term strategy to increase engagement with fully automated programs is to work toward a detailed understanding of attrition in fully automated contexts. Attrition could then be reduced by creating more engaging Internet sites, rather than through costly cohort maintenance procedures.

In predominantly single language, single country samples, previously identified predictors of attrition from internet interventions include younger age, lower levels of education, unmarried status and high levels of symptom severity (Bull et al., 2004; Couper et al., 2007; Buller et al., 2008; McKay et al., 2008; Melville et al., 2010). Lower expectations and confidence that the intervention will work has also been found to predict dropout (McKay et al., 2008; Geraghty et al., 2010). As Internet interventions are designed to target larger, multinational samples for global health problems in efforts to reduce health disparities worldwide, it is important to identify whether any consistent patterns in attrition emerge. Although the increase in variance from multinational samples may make the identification of significant predictors of attrition more difficult, studying attrition from large international automated interventions is critical both for ecological validity of such investigations as well as for the continual improvement of such interventions. Finding significant predictors of attrition from large international samples would demonstrate that even with the likely increase in variance, behavioral disengagement from trials of international Internet health interventions is not a random phenomenon, but systematic. The identification of consistent predictors of attrition from these large trials would enable a ‘science of attrition’ (Eysenbach, 2005) broad enough to be generalizable to international eHealth campaigns. These predictors can then be investigated further and distilled into manipulations of intervention content with the aim of reducing attrition without adding human contact, which is crucial for sustainability of such interventions. Developing internet interventions with tested methods for encouraging and motivating sustained participation in the trial as well as the desired behavior change (e.g. smoking cessation) would ensure that automated Internet interventions can be evaluated on the basis of their effect (rather than attrition, Leykin et al., 2012), iteratively improved and achieve maximal global impact.

THE PRESENT STUDY

This study models attrition in a secondary data analysis of an international, dual-language (Spanish and English) Internet smoking cessation intervention. Our aim is to examine and identify predictors of attrition that are likely to inform the development of future large-scale global interventions. We examine FU attrition from two cohorts of an online intervention: one employing only automated FU and the other employing live FU strategies, as described below.

Motivation is likely to be a critical factor underlying attrition. We examined the association between attrition and confidence in ability to quit, since earlier research found that confidence predicted retention in an online smoking cessation trial (McKay et al., 2008). We also examined the association between attrition and motivation, using self-selected delay of the quit date as an indicator of the eagerness to begin the quit attempt. Lower expectancies and motivation for the target behavior of the intervention (i.e. quitting smoking) might also lead to greater disengagement from related sub-goals (i.e. fully participating in a trial); we therefore hypothesized that low confidence and selecting a longer delay before quit date would be related to higher FU attrition. Closely related to motivation, we hypothesized that higher nicotine dependence and depression status would both be related to FU attrition. Depression may negatively influence volition, while high dependence will increase the difficulty of quitting, increasing the likelihood of disengagement. Positive affect was also explored as previous research has shown that positive affect can be associated with retention (Bränström et al., 2010).

We explored demographic variables previously found to predict attrition including age, and education, gender, race and marital status (Bull et al., 2004; Couper et al., 2007; Buller et al., 2008; McKay et al., 2008; Melville et al., 2010). In addition, we explored language (Spanish vs. English), which was an integral aspect of this intervention. An association between language and attrition would suggest limits to translating and offering automated interventions in different languages without a deeper exploration of associated cultural factors that may have an influence on attrition. Finally, we examined factors related to the intervention itself, such as whether participants were randomized to conditions with more frequent automated email contact and their baseline frequency of email use, as well as factors related to study entry, such as date of entry into the study—rapid evolution of the Internet led us to hypothesize that the ‘aging’ of the site might reduce its appeal for latter enrollees.

METHOD

The data we present are from two cohorts of an Internet-based self-help smoking cessation intervention. In one cohort, FU was fully automated (n= 16 430, estimated quit rate: 21–30%, see Leykin et al., 2012); participants were sent emails with a link to the online FU surveys (referred to as ‘automated FU cohort’ for brevity). The other cohort added live FU (n= 1000, quit rate: 20%, see Muñoz et al., 2009) in the form of personal emails and phone calls if the participants did not respond to the automated email FUs (referred to as ‘live FU cohort’). The intervention was based on a smoking cessation intervention originally developed for Spanish-speaking Latinos (Pérez-Stable et al., 1991; Muñoz et al., 1997) and later adapted for English- and Spanish-speaking Internet users (Muñoz et al., 2006, 2009). This research was approved by the Institutional Review Board of the University of California, San Francisco.

Eligibility

Eligible participants were 18 years or older, smoked 5+ cigarettes per day, used email at least once weekly and intended to quit in the next month.

Recruitment

Participants were recruited via Google Ads, radio or television referrals, online resources such as newsgroups and magazines, and from organic searches on Google and other search engines. All participants enrolled in the study online by completing consent procedures, baseline questionnaires, reporting on cigarettes smoked on three additional days within 1 week and setting their initial quit date. Participants were then randomized to one of four Internet intervention conditions, described below, stratified within each language by gender and baseline depression status. The first 500 Spanish and 500 English speakers to be randomized comprised the live FU cohort (Muñoz et al., 2009); all subsequent randomized participants comprised the automated FU cohort (n= 16 430). FUs occurred at 1, 3, 6 and 12 months after their self-selected quit date; the participants were able to access the site for the duration of the study.

Intervention conditions and FU methods

Randomization conditions are described in detail in Muñoz et al. (2009) Condition 1 (basic) consisted of a static smoking cessation guide (Pérez-Stable et al., 1991), while Conditions 2–4 incrementally added intervention elements. Thus, Condition 2 added a series of reminder emails individually timed to the participants' quit date, inviting the participant to return to the site, (see Muñoz et al., 2006), Condition 3 further added 8 lessons on mood management to help individuals cope with mood difficulties that might arise during the quit attempt and Condition 4 further added a virtual group—an asynchronous bulletin board for peer support and discussions.

At 1-, 3-, 6- and 12-months, automated emails with links to online FU surveys were sent. Participants received up to three automated emails within 1 week, with surveys available for a 4–6 week window at each FU point. For participants in the live FU cohort who did not respond to automated emails, live FU was attempted with research assistants sending personal emails and calling up to 10 times; answers to smoking outcome questions were requested as a minimal response.

Measures

Attrition

Attrition was examined separately for the entire FU period and for response to the 1-month FU. Participants were considered completely lost to FU if they did not respond to any smoking outcome question for any FU point, and were considered lost to 1-month FU if they did not provide smoking outcome data for this first FU period. Analyses for the 1-month FU were undertaken in order to identify which variables emerged as early predictors of attrition, given high attrition rates that often occur in the very early phases of trials (Eysenbach, 2005; Christensen et al., 2006).

Demographic variables

A demographic questionnaire contained questions assessing participants' age, gender, race/ethnicity, education and marital status.

Motivation/confidence

A single 10-item question, ‘How confident are you that you will be able to stop smoking?' assessed participants' confidence in ability to quit. Delay in selecting a quit date was assessed by creating a variable representing the number of days between the date participants were asked by the intervention site to set a quit date and the future quit date they selected (this had to be within 30 days).

Dependence

A smoking questionnaire assessed the number of cigarettes smoked per day, whether this was a first quit attempt, smoking exposure and nicotine dependence level. The Fagerström Test for Nicotine Dependence (Heatherton et al., 1991) is a commonly used 6-item nicotine dependence scale, with a range from 0 to 10.

Depression

The major depressive episode (MDE) Screener (‘Mood Screener’, Muñoz, 1998) is a self-report screening instrument assessing for the presence of the nine symptoms of current and past MDEs according to the Diagnostic and Statistical Manual of Mental Disorders-IV (APA, 1994), as well as for significant impairment in functioning (Criterion C). This instrument has been shown to have good agreement with the Primary Care Evaluation of Mental Disorders scales and clinician administered interviews (Spitzer et al., 1994; Muñoz et al., 1999; Vázquez et al., 2008).

The Center for Epidemiological Studies – Depression scale (CES-D; Radloff, 1977) is a widely used 20-item self-report depression symptom level measure. The four positive items on the scale were used as a measure of positive affect.

Analysis

Binary logistic regressions were used to examine the association between attrition and variables representing motivation, as well as baseline demographics, smoking, depression status and symptoms, intervention exposure and entry. Differences in recruitment patterns, FU procedures and consequent FU rates indicated that nearly identical, yet separate analyses were warranted for the automated and live FU cohorts. For example, since the rapid recruitment rate for the live FU cohort (all 1000 recruited in 13 weeks) effectively eliminated any meaningful ‘age of site’ effects between the first and last enrolled participants, week of study entry was excluded from analysis for the live FU cohort but not the automated FU cohort. Separate analyses were also used to discover which associations with attrition would persist and which would be attenuated with conditional live FU. Two sets of binary logistic regressions were conducted, one analyzing those completely lost to FU and another analyzing those lost at the 1-month FU point. All variables were entered simultaneously in a single block into the respective logistic regressions to evaluate the hypothesized relationships in the presence of all potential predictors. To minimize the risk of spurious associations, significance levels were set to p < 0.001 for the automated FU cohort due to the large sample size and p < 0.01 for the live FU cohort.

RESULTS

Participant characteristics

Participants in the automated FU cohort represented over 150 countries and territories, with over two-thirds of the English speakers predominantly from the USA (n= 1250), India (n= 358), South Africa (n= 306), the UK (n= 181) and Australia (n= 140), and over 75% of the Spanish speakers predominately from Spain (n= 4341), Argentina (n= 2514), Mexico (n= 2100) and Chile (n= 1061). Represented countries were very similar for the live FU cohort and are described in detail elsewhere (Muñoz et al., 2009).

Participant characteristics for the automated FU cohort (n= 16 430) and the live FU cohort (n= 1000) are described in Table 1. Since the live FU cohort was designed to be the first 500 Spanish speakers and the first 500 English speakers to be randomized, they differed from the fully automated cohort in the proportion of English speakers (50% live vs. 20.3% automated) and the week of enrollment into the study [mean week of enrollment, 3.2 (SD = 4.1) vs. 85.3 (SD = 44.1)]. The automated FU cohort self-identified as mestizo more often and Asian less often than the live FU cohort, had less education, less broadband access (55.2 vs. 65.3%), checked email fewer days per week [5.7 (1.8) vs. 5.9 (1.6)], were more likely to meet criteria for a current MDE (17.2 vs. 12.9%) and less likely for past MDE only (13.9 vs. 17.4%), and were more likely to be making their very first quit attempt (15.3 vs. 11.3%). The two cohorts did not differ however in age, proportion who were married or partnered, CES-D score, nicotine dependence, cigarettes smoked per day and exposure to smoking at home or work.

Table 1:

Characteristics of the automated and live FU cohorts

Variable	Automated FU	Conditional live FU	P-value
Demographics	n = 16 430	n = 1000
Age	36.2 (10.7)	37.1 (11.3)	0.012
Sex (male, %)	52.8	55.0	0.185
Language (Spanish, %)	79.7	50.0	*
Partnered (%)	53.9	54.1	0.879
Education level (%)			<0.001
High school or less	22.9	17.1
Some college	36.7	39.5
College	28.4	28.7
Masters or more	12.0	14.7
Race (%)			<0.001
Asian	3.8	8.2
White	68.7	70.0
African/African American	1.2	1.4
Mestizo	16.4	10.8
Other	10.0	9.7
Smoking
Fagerstrom score	5.20 (2.5)	5.23 (2.5)	0.762
First quit attempt (%)	15.3	11.3	0.001
Cigarettes per day	19.6 (10.1)	19.8 (10.1)	0.608
Smoke exposure (%)	69.0	66.0	0.047
Depression related
Positive CESD subscale	7.1 (3.2)	7.3 (3.1)	0.037
MDE history (%)			<0.001
No history	68.9	69.6
Past only	13.9	17.4
Current	17.2	12.9
Intervention related
Days per week check email	5.7 (1.8)	5.9 (1.6)	<0.001
Week of enrollment	85.3 (44.1)	3.2 (4.1)	*
Broadband (%)	55.2	65.3	<0.001

Note: p values represent t-tests for means and Fisher's exact chi-squared tests for proportions.

*Differences directly due to assigning first 500 Spanish and first 500 English speakers randomized.

Predictors of complete loss to FU

Nearly half of all participants (48.1%) were completely lost to FU in the automated FU cohort, compared with 10.8% of participants in the live FU cohort. For the automated FU cohort, the overall binary logistic regression model for complete loss to FU was significant (χ² = 790.27, df = 26, p < 0.001). As illustrated in Table 2, complete loss to FU in the automated FU cohort was associated with younger age, male sex, having a high school or some college education compared with master's level or above, greater nicotine dependence and a first quit attempt. In addition, greater delay of the self-selected quit date and lower self-reported confidence for quitting were also associated with complete loss to FU. Compared with those in the basic condition, those randomized to receive reminder emails were more likely to be lost to FU, while those with access to the mood condition were more likely to be lost to FU compared with those receiving reminder emails alone. Finally, those enrolling later in the study were slightly more likely to be lost.

Table 2:

Logistic regression predicting participants completely lost to FU, automated FU cohort (n = 16 430)

Variables	Odds ratio	95% confidence interval for odds ratio	Significance*
Age at enrollment	0.97	(0.969–0.976)	<0.001
Sex
Female	Ref		Ref
Male	1.30	(1.22–1.39)	<0.001
Education level			<0.001
Masters or higher	Ref		Ref
Bachelors degree	1.08	(0.97–1.21)	0.177
Some college	1.23	(1.10–1.37)	<0.001
High school or less	1.27	(1.13–1.43)	<0.001
Language
Spanish	Ref		Ref
English	1.14	(1.04–1.25)	0.005
Race			0.077
White	Ref		Ref
Asian	0.93	(0.77–1.12)	0.453
African/African American	1.23	(0.90–1.68)	0.196
Mestizo	0.89	(0.81–0.98)	0.014
Other	0.97	(0.87–1.09)	0.649
Relationship status
Single	Ref		Ref
Married/living with partner	1.04	(0.98–1.12)	0.215
Nicotine dependence (Fagerstrom)	1.04	(1.02–1.05)	<0.001
Quit attempt
Not first attempt	Ref		Ref
First quit attempt	1.19	(1.09–1.30)	<0.001
Cigs per day	1.01	(1.003–1.012)	0.001
Smoke exposure
None	Ref		Ref
At work, at home, or both	1.08	(1.01–1.16)	0.028
MDE history			0.269
No history	Ref		Ref
Past history only	0.96	(0.88–1.06)	0.446
Current episode	1.06	(0.97–1.17)	0.214
Positive CESD items	1.00	(0.99–1.01)	0.645
Days quit delayed	1.02	(1.02–1.02)	<0.001
Confidence (10 = complete)	0.97	(0.95–0.98)	<0.001
Condition			<0.001
Basic vs. basic + email, +mood, +group	1.32	(1.25–1.40)	<0.001
Basic + email vs. +mood, +group	1.11	(1.05–1.17)	<0.001
Mood vs. + group	1.04	(0.99–1.08)	0.118
Week of enrollment	1.002	(1.001–1.003)	<0.001
Days check email	1.01	(0.99–1.03)	0.412
Connection
Broadband	Ref		Ref
Other	0.96	(0.89–1.02)	0.195

*Significance level set at p < 0.001, given a large sample size.

For the live FU cohort, however, the binary logistic regression model was not significant for complete loss to FU (χ² = 37.20, df = 25, p = 0.055), suggesting that live FU can successfully mitigate systematic differences introduced in the resulting sample when only automated FU is used.

Loss to 1-month FU

At 1 month, 61.4% of participants in the automated FU cohort were lost to FU, 78.4% of whom were eventually lost to FU completely. The overall binary logistic regression for the automated FU cohort was significant (χ² = 984.76, df = 26, p < 0.001). This analysis indicates that the above patterns of predictors for complete loss to FU begin to appear as early as the 1-month FU point (see Table 3), with younger age, male sex, greater nicotine dependence, greater quit delay, less confidence, week of study enrollment, email reminders and mood lessons again predicting higher attrition. There was a trend toward education predicting loss at 1-month FU at this early point, although it did not meet our specified significance level of <0.001 and first quit attempt did not predict attrition at this stage.

Table 3:

Logistic regression predicting participants lost to 1-month FU, automated FU cohort (n =16 430)

Variables	Odds ratio	95% confidence interval for odds ratio	Significance*
Age at enrollment	0.97	0.97–0.97	<0.001
Sex
Female	Ref		Ref
Male	1.38	(1.29–1.48)	<0.001
Education level			0.001
Masters or higher	Ref		Ref
Bachelors degree	1.04	(0.93–1.17)	0.447
Some college	1.17	(1.05–1.30)	0.006
High school or less	1.22	(1.08–1.37)	0.001
Language
Spanish	Ref		Ref
English	1.13	(1.02–1.24)	0.015
Race			0.149
White	Ref		Ref
Asian	0.97	(0.80–1.19)	0.782
African/African American	1.38	(0.98–1.95)	0.063
Mestizo	0.92	(0.84–1.01)	0.092
Other	1.00	(0.89–1.12)	0.971
Relationship status
Single	Ref		Ref
Married or living with partner	1.05	(0.98–1.12)	0.207
Dependence (Fagerstrom)	1.04	(1.02–1.05)	<0.001
Quit attempt
Not first attempt	Ref		Ref
First quit attempt	1.12	(1.02–1.23)	0.015
Cigs per day	1.01	(1.00–1.01)	0.004
Smoke exposure
None	Ref		Ref
At work, at home, or both	1.09	(1.02–1.18)	0.013
MDE history			0.033
No history	Ref		Ref
Past history only	0.97	(0.88–1.07)	0.514
Current episode	1.12	(1.02–1.24)	0.019
Positive CESD items	0.99	(0.98–1.01)	0.339
Days quit delayed	1.02	(1.02–1.03)	<0.001
Confidence (10 = complete)	0.96	(0.95–0.98)	<0.001
Condition			<0.001
Basic vs. basic + email, +mood, +group	1.50	(1.42–1.59)	<0.001
Basic + email vs. +mood, +group	1.20	(1.14–1.27)	<0.001
Mood vs. + group	1.02	(0.98–1.07)	0.334
Week of enrollment	1.002	(1.001–1.003)	<0.001
Days check email	1.00	(0.98–1.02)	0.749
Connection
Broadband	Ref		Ref
Other	0.97	(0.90–1.04)	0.376

*Significance level set at p < 0.001, given a large sample size.

The binary logistic regression model of loss at the 1-month FU for the live FU cohort was significant (χ² = 56.88, df = 25, p < 0.001). However, only younger age was associated with loss to FU at 1 month (OR = 0.97, 95% CI 0.96–0.99, p = 0.001).

DISCUSSION

Our results highlight the reach and potential of the Internet to deliver automated evidence-based smoking cessation interventions, with over 16 000 participants enrolling to take part from over 150 countries worldwide. The willingness of participants to take the steps necessary to enroll (such as signing informed consent, completing a long baseline questionnaire and returning to the site on 3 separate days to enter a number of cigarettes smoked per day) demonstrates that Internet interventions are seen as potentially helpful resources to facilitate behavior change at the international level. An increased understanding of how to maintain participants' engagement with materials and trial procedures will allow this initial volition to be maintained leading to more complete data capture, and full evaluation of interventions. Automated Internet interventions often report full attrition (no FU data collected) of up to 80% (Eysenbach, 2005; Christensen et al., 2006). Over 50% of participants in our fully automated sample provided some level of FU data over the year study period. Without overt attempts to curtail attrition, achieving a degree of FU response in more than half our sample suggests that with direct targeting and tailoring on specific attrition-related factors, this level of retention is likely to be increased further.

In the automated FU cohort, those individuals not providing any FU data had a lower level of education, were more dependent on nicotine, less confident in their ability to quit and had chosen longer delays before quit dates. All these factors are indicative of those who need help the most. Ideally, these individuals would be able to access face-to-face smoking cessation services. However, this service is unlikely to be available to many, and for these individuals our intervention may have been the only readily accessible quit aid. For these individuals, modifications of our Internet interventions may have improved retention. For instance, modules aimed at increasing confidence and reducing ambivalence could be added, as well as automated algorithms that evaluate the level of dependence at baseline and provide additional materials, support and encouragement. With regard to reducing attrition in those with lower levels of education, interventions could be personalized by selecting a preferred format, for instance, video or audio instead of text (Clarke et al., 2002).

The development and testing of content specifically designed to reduce attrition is a new research area; results from this investigation are fundamental in order to provide the basis for hypothesis generation regarding potential adaptations. For example, Bandura's self-efficacy theory (Bandura, 1997) and motivational interviewing (MI, Miller and Rollnick, 2002) may provide frameworks for adaptations to increase confidence and reduce ambivalence as mentioned above. Modelling or vicarious experience is reported to be an important way to increase self-efficacy (Bandura, 1997), thus, including modules with vignettes (written or video) that show or explain how a range of individuals not only successfully quit smoking, but how they used the internet intervention to do so, might increase self-efficacy and thus reduce attrition. Additionally, taking a non-directive approach to encouraging change in the intervention text, and providing opportunities for individuals' to record and focus on their own motivations for change might help reduce ambivalence according to MI. These suggestions are speculative, based on our understanding of human motivation and mechanisms of change. However, these ideas should be approached and studied empirically.

The absence of depression status and language as significant predictors of attrition in our study is encouraging. Contrary to our predictions, depression did not appear to have increased the likelihood of non-completion across all follow-up points while positive affect was also unrelated to attrition. Language was likewise not a predictor of attrition. This suggests that some attrition-related factors might be universal. Thus, if the present research is replicated with additional languages, attrition-reducing modifications are likely to be applicable to behavior change Internet interventions applied in multiple countries, in multiple languages.

Further factors we have identified highlight the need for increased systematic research on attrition. A key area for future research is the influence of intervention variables on attrition. For example, we found that participants in conditions that received multiple emails as part of their intervention and individuals who were given access to a mood intervention, were more likely to be lost to FU. One possibility is that those who received a higher volume of study-related emails may have experienced ‘email fatigue’ reducing novelty of study-related emails and increasing the chances of FU emails being ignored. While mood lessons were made available in two conditions, randomization to this feature may have been unwelcome among some smokers and may have contributed to greater disengagement due to a perception of irrelevance. Greater engagement, and consequentially reduced attrition, however, might be observed if participants are permitted to choose intervention elements themselves. Finally, age (younger) and gender (male) predicted attrition. Although the relation of age is the most consistent with previous attrition research, it is also the least understood. Basic qualitative research is needed to begin to develop theoretical models for this association, so modifications can be developed and implemented in future Internet interventions. Ultimately, the ability to evaluate large-scale Internet interventions will hinge on increasing engagement, thereby enabling more complete FU.

While attrition research develops, incorporating design features such as following up a representative sub-sample with live methods (telephone calls) may allow extrapolation of intervention effectiveness from smaller live FU cohorts to full samples of automated trials. We found that when following-up a sub-sample with live methods, 89.2% provided data at least once during the year. This high FU rate removed many of the biases identified in the fully automated sample, as the cohort maintenance sample providing data were more representative of those originally randomized.

We note some limitations to our study. Our target health problem was smoking, and languages were English and Spanish. It is as yet unclear whether the identified predictors would generalize to other health problems and languages. Nonetheless, consistency with previous research on predictors of attrition from interventions with a variety of health targets indicates that our identified predictors may be relatively stable. We chose to focus on predictors that we felt could theoretically drive attrition. Additional exploratory approaches (such as classification and regression tree analysis) may determine other predictors. The results of this study were based on secondary data analyses. Future research should approach this problem experimentally, varying specific variables in an ABAB design, that is, where the attrition is monitored without specific modifications (A), monitored again with modifications in place (B), with modifications removed (A), and modifications implemented again (B), or similar designs, to determine whether factors can reliably increase adherence.

To conclude, as access to the Internet continues to grow across the world, Internet behavior change interventions have the potential to become a key tool in the delivery of broad public health initiatives. Systematic attrition analyses, such as those presented here, allow researchers to immediately begin modification of large-scale Internet intervention designs in order to increase retention, and allow those considering utilization of the Internet in future research an indication of some of the key challenges they are likely to face.

FUNDING

This work was supported by the Tobacco Related Disease Research Program, the University of California, Office of the President, the Google Adwords Grants Program, the Brin/Wojcicki Foundation, the National Cancer Institute (NCI) for Redes en Acción (U01CA86117) and by NIMH grants 5T32MH018261-27 and 5K08MH091501.