Abstract
Background
With the growing use of Internet-based interventions, strategies are needed to encourage broader participation. This study examined the effects of combinations of monetary incentives and mailing characteristics on enrollment, retention, and cost effectiveness for an online health program.
Methods
In 2004, a recruitment letter was mailed to randomly selected Midwestern integrated health system members aged 21–65 and stratified by gender and race/ethnicity; recipients were randomly pre-assigned to one of 24 combinations of incentives and various mailing characteristics. Enrollment and 3-month retention rates were measured by completion of online surveys. Analysis, completed in 2005, compared enrollment and retention factors using t tests and chi-square tests. Multivariate logistic regression modeling assessed the probability of enrollment and retention.
Results
Of 12,289 subjects, 531 (4.3%) enrolled online, ranging from 1% to 11% by incentive combination. Highest enrollment occurred with unconditional incentives, and responses varied by gender. Retention rates ranged from 0% to 100%, with highest retention linked to higher-value incentives. The combination of a $2 bill prepaid incentive and the promise of $20 for retention (10% enrollment and 71% retention) was optimal, considering per-subject recruitment costs ($32 enrollment, $70 retention) and equivalent enrollment by gender and race/ethnicity.
Conclusions
Cash incentives improved enrollment in an online health program. Men and women responded differently to mailing characteristics and incentives. Including a small prepaid monetary incentive ($2 or $5) and revealing the higher promised-retention incentive was cost effective and boosted enrollment.
INTRODUCTION
Internet-based health interventions have increased in recent years,1,2 offering economical and sophisticated programs to a large and diverse group of people, along with a concomitant growth in RCTs exploring the efficiency of online interventions. Little is known about effective ways to recruit participants for RCTs or behavioral interventions online.3 Two common strategies include using the Internet for recruitment4,5 and targeted recruitment using list-based mailed invitations.5,6 Few recruitment strategies have been evidence-based, relying rather on case studies and opinion rather than on sound evidence.3 In contrast, a large body of literature describes strategies for recruiting respondents for surveys,7 focusing on the use and effectiveness of incentives for boosting response rates.8–13 Specifically, prepaid or unconditional monetary incentives are shown to be most effective, while cash exceeds other nonmonetary incentives, and increasing the amount of the incentive increases response, up to a point.14–16
A token cash incentive in the mailed invitation may be effective both in drawing attention to the request and in invoking norms of reciprocity, leading to increased compliance with the request.17 Those less interested in the study may be more responsive to incentives.18,19 Other studies have examined the effectiveness of varying the invitation mailing, including stamped versus metered mail,20–22 the content of the cover letter,7,23 and other response-inducing strategies.
This study compares the effectiveness and costs of recruiting and retaining subjects in an online health behavior program using varied incentive combinations and mailing characteristics. Based on prior work,6 the goal was to achieve a 10% enrollment rate with diversity by gender and race/ethnicity, with low costs, and with the ability to compare prepaid incentives to promised or conditional incentives. Results were expected to vary by gender and race/ethnicity, reflecting differential access to the Internet and interest in the program topic. To our knowledge, no prior studies have explored the differential response by demographic subgroups to such recruitment or retention strategies, particularly in relationship to an online program.
Methods and Materials
This study assessed the feasibility of attaining adequate enrollment and retention without telephone contact, preliminary to a larger Cancer Research Network (CRN) multi-site, web-based nutrition intervention trial, Making Effective Nutritional Choices for Cancer Prevention (MENU). The IRBs at the collaborating institutions approved the project, and all participants provided informed consent.
Study Population
In 2004, adults aged 21–65 were randomly sampled from a large Midwest health system with an affiliated HMO, using a stratified, list-based sample. Eligibility, utilizing health plan data, included 12-month enrollment, age, and no contraindications for changing fruit and vegetable intake—specifically, no active cancer treatment, gastroparesis diagnosis, or anticoagulant medication use. Required Internet access and at least weekly e-mail use were assessed by the eligibility survey.
Study Protocol
Before recruitment, subjects were randomly preassigned to one of 24 incentive groups, stratified by gender and race/ethnicity (African American versus all others) based on administrative data (Table 1). Each group received either no incentive, a prepaid (unconditional) incentive, or a promised (conditional) incentive for enrolling in the study, combined with either no incentive, a prepaid incentive, or a promised incentive for retention (Table 2). All incentives were monetary and paid in cash.
Table 1.
Descriptive characteristics of invitees, enrollees and those retaineda
| Invitedb (N=12,889) | Enrolledc (n=531) | Retainedd (n=297) | |
|---|---|---|---|
| ADMINISTRATIVE DATA | |||
| Age mean (SD) | 43.0 (11.4) | 45.9 (11.2) | 46.0 (11.8) |
| Race/gender (%) | |||
| African American | 50 (n=5917) | 43 (n=228) | 38 (n=112) |
| Men | 25 (n=2992) | 15 (n=81) | 39 (n=44) |
| Women | 25 (n=2925) | 28 (n=147) | 61 (n=68) |
| White/other | 50 (n=5841) | 57 (n=303) | 62 (n=185) |
| Men | 25 (n=2952) | 22 (n=116) | 36 (n=66) |
| Women | 25 (n=2885) | 35 (n=187) | 64 (n=119) |
| SURVEY DATA | |||
| Highest level of education (%) | |||
| ≤ High school | 18 | 14 | |
| Vocational | 7 | 7 | |
| Some college/associate degree | 38 | 38 | |
| College/post college | 37 | 41 | |
| Income(%) | |||
| <$30,000 | 14 | 15 | |
| $30–49,999 | 25 | 21 | |
| $50–69,999 | 21 | 21 | |
| $70,000+ | 40 | 35 | |
| Married/partner (%) | 70 | 69 | |
Some variables had missing data; therefore, columns do not equal total.
Invited was defined as anyone who was sent the initial recruitment letter.
Enrolled was defined as any eligible invitee who completed the baseline survey.
Retained was defined as an enrollee who completed the 3-month follow-up survey.
TABLE 2.
Distribution of enrollment stratified by gender and ethnicity, retention, and costs by incentive conditiona
| Incentive group numberb | Enrollment incentivec ($) | Retention incentivec ($) | Letters sent (n) | Enrollment response rate (%) n=531 | AA men enrolled (%) n=81 | AA women enrolled (%) n=81 | Other men enrolled (%) n=116 | Other women enrolled (%) n=187 | Retention overall (%) n=297 | Enrollment cost per subject ($) | Retention cost per subject ($) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 0 | 0 | 768 | 2.2 | 2 | 2 | 1 | 8 | 24 | 45 | 196 |
| 2 | 0 | 5.00 | 768 | 3.1 | 2 | 4 | 2 | 5 | 54 | 32 | 70 |
| 3 | 0 | 10.00 | 768 | 2.7 | 1 | 4 | 1 | 5 | 48 | 37 | 90 |
| 4 | 0 | 20.00 | 768 | 2.9 | 0 | 4 | 3 | 4 | 45 | 35 | 100 |
| 5 | 1.00 | 0 | 648 | 2.8 | 2 | 4 | 1 | 4 | 44 | 72 | 164 |
| 6 | 1.00 | 5.00 | 648 | 2.6 | 3 | 4 | 1 | 2 | 41 | 76 | 200 |
| 7 | 1.00 | 10.00 | 648 | 4.9 | 4 | 4 | 5 | 7 | 56 | 41 | 85 |
| 8 | 1.00 | 20.00 | 648 | 6.8 | 4 | 7 | 7 | 10 | 57 | 29 | 75 |
| 9 | 2.00 | 0 | 648 | 3.5 | 2 | 1 | 6 | 6 | 30 | 85 | 281 |
| 10 | 2.00 | 5.00 | 648 | 5.1 | 5 | 6 | 4 | 5 | 55 | 59 | 119 |
| 11 | 2.00 | 10.00 | 648 | 9.4 | 7 | 9 | 10 | 11 | 56 | 32 | 70 |
| 12 | 2.00 | 20.00 | 648 | 9.7 | 8 | 9 | 9 | 12 | 71 | 31 | 66 |
| 13 | 5.00 | 0 | 120 | 7.5 | 3 | 7 | 10 | 10 | 22 | 80 | 365 |
| 14 | 5.00 | 5.00 | 120 | 5.8 | 3 | 10 | 7 | 3 | 14 | 103 | 762 |
| 15 | 5.00 | 10.00 | 120 | 6.7 | 0 | 7 | 13 | 7 | 50 | 90 | 193 |
| 16 | 5.00 | 20.00 | 120 | 10.8 | 3 | 13 | 7 | 20 | 69 | 55 | 102 |
| 17 | 10.00 | 0 | 768 | 2.7 | 2 | 3 | 3 | 3 | 33 | 48 | 146 |
| 18 | 10.00 | 5.00 | 768 | 3.1 | 1 | 4 | 3 | 5 | 79 | 43 | 62 |
| 19 | 10.00 | 10.00 | 768 | 2.7 | 0 | 4 | 2 | 6 | 86 | 48 | 68 |
| 20 | 10.00 | 20.00 | 769 | 4.8 | 3 | 6 | 3 | 7 | 68 | 32 | 70 |
| 21 | 20.00 | 0 | 120 | 1.7 | 0 | 0 | 0 | 7 | 50 | 81 | 164 |
| 22 | 20.00 | 5.00 | 120 | 0.8 | 0 | 0 | 0 | 3 | 0 | 141 | — |
| 23 | 20.00 | 10.00 | 120 | 5.0 | 0 | 10 | 3 | 7 | 83 | 41 | 61 |
| 24 | 20.00 | 20.00 | 120 | 5.8 | 3 | 10 | 7 | 3 | 100 | 38 | 60 |
Total letters sent: N=12,289; overall enrollment n=531; 3-month follow-up survey completion n=297.
Groups 13–16 and 21–24 were in Wave 1 only. Groups 1–4 and 17–20 were in both Waves 1 and 2. Remaining groups were in Wave 2 only.
Cash incentives: unconditional=$1, $2, and $5; conditional=$10 and $20.
A single, mailed recruitment letter, printed on standard-size institutional letterhead, invited enrollment in a free, online program that encouraged the eating of more vegetables. The letter described eligibility and enrollment information, and displayed the study website and a unique pass code. The letter also revealed any incentive conditions for both enrollment and retention, and included any prepaid incentive and a descriptive insert. Conditional incentives were mailed following enrollment and completion, or simply the completion, of the follow-up survey. Participants were unaware of incentive structures for other invitees. Based on prior studies7,21 and confirmed by study focus groups, standard-size, business-style envelopes were used, and the envelope displayed the health system’s logo.
Enrollment and all surveys were completed online. Enrollment was defined as completion of the eligibility survey (11–25 questions, varying with tailoring), the giving of informed consent, the verification of a functional e-mail address, and the completion of the baseline survey (38 questions) within a 28-day enrollment window. Automated e-mail reminders, including an activated website link, announced the availability of two tailored newsletters adapted from the Better Health study,24 which provided strategies for increasing vegetable intake. Newsletters appeared following enrollment and were available during the 3-month study. All materials were limited to English and not tailored to ethnicity.
Retention was defined as completing the follow-up survey (57 questions) 3 months after enrollment. An “alert” letter that included any prepaid incentive was mailed prior to the follow-up survey completion date. Participants had 38 days to complete the survey. Automated e-mail notices were sent every 3 or 4 days to nonresponders. The letter and e-mail reminders described any retention incentive amount.
Incentive combinations
The 24 incentive combinations (Table 2) were evaluated incrementally in two mailing waves over 10 weeks. Six enrollment incentive options of $0; prepaid $1, $2 bill, or $5; or promised $10 or $20 were coupled with each of four retention incentives of $0, prepaid $5, or promised $10 or $20. Incentives beyond $20 were not considered because of cost considerations.
In the first wave, 120 letters were sent to people assigned to one of 16 incentive combinations, Groups 1–4 and 13–24 (Table 2). The $5 prepaid incentive (Groups 13–16) outperformed the other enrollment conditions ($0, $10 promised, or $20 promised), suggesting the recruitment advantage of a small prepaid incentive. Wave 2 tested the effectiveness of smaller, potentially more cost-effective $1 or $2 prepaid incentives, based on Doody and colleagues9 who found a $2 bill to be most effective and often used in mail surveys. The $20 promised enrollment incentive (Groups 21–24) was discontinued due to low and disproportionate enrollment rates and high cost (a maximum of $141 per enrollee). Eight incentive groups (Groups 13–16 and 21–24) were discontinued and eight new groups (Groups 5–12) were added, with 648 letters sent to remaining assigned Groups 1–12 and 17–20.
Before Wave 2, the recruitment letter was revised to a “scan-friendly” reading format that prominently displayed enrollment information. A colorful 3″ × 8″ flier replaced the previous insert. A Post-it® note that displayed the website address, a strategy shown to be effective in prior research,25 was included in one third (35%) of the mailings. Half of the mailings used postage stamps, and the other half used metered postage.
Statistical Analysis
Although a sample-size calculation was done in planning the study, the final sample sizes were essentially a function of the project’s capacity to mail equal numbers of invitation letters per incentive condition within a limited time frame.
T tests were used for age differences and chi-square tests for other variables. Cost per enrollee was the sum of any advance or promised incentive payment, plus mailing costs ($1 per letter sent), divided by the number enrolled. Retention costs summed the expenses for enrollment, the mailed follow-up survey alert letter, and any prepaid or conditional retention incentive, plus the cost of the conditional incentive mailing, divided by the number completing the follow-up survey.
Two sets of multiple logistic regression analyses assessed the probability of enrollment and of retention. Both enrollment and retention incentives, and subject and mailing characteristics, were included in each of the models. All two-way interactions involving race/ethnicity and gender were tested, using backward stepwise elimination with a plan to use stratified analyses as necessary. Age was represented using indicator variables for each decade. Stratified analyses were performed when significant interactions were observed, and all tests were two-tailed. Analyses occurred between 2004 and 2005, using SAS 9.1.
Results
Of 12,289 recruitment letters mailed, 531 subjects enrolled (4.3%). In Wave 1, 87 subjects enrolled out of 1920 letters mailed (4.5%). In Wave 2, 444 subjects enrolled of 10,369 letters mailed (4.3%). Based on health plan data, more women and white/other ethnic groups enrolled (5% women, 3% men [p<0.001]; 5% white/other, 4% African American [p<0.001]), and enrollees were statistically older than invitees (Table 1) (enrollees’ mean age=45.9, SD=11.2; non-enrollees’ mean age=43.0, SD=11.4, p<0.001).
Of those enrolled, more were women (63%); more were in the white/other ethnic group (57%) than in the African-American group (43%); more were married or living with a partner (70%); more reported “some college” or higher education (75%), had a household income of $50,000 or greater (61%), and reported being “very comfortable” with Internet use (63%). One third (38%) rated their personal health as “very good” or “excellent.”
Enrollment varied by incentive group, ranging from 1% to 11% (Table 2). Enrollment cost per participant ranged from $29 (Group 8) to $141 (Group 22) across incentive combinations. Those groups that were offered no enrollment incentive (Groups 1–4) had the lowest enrollment rates (2%–3%) and costs ($32–$45 per participant), compared to the monetary incentive groups. The $5 prepaid incentive (Groups 13–16) had the best overall enrollment rates (6%–11%) and the highest costs per participant ($102 for enrollment, $762 for retention). The highest enrollment rate overall, 11% (n=58), was found in Group 16 ($5 enrollment, $20 retention), although enrollment was unbalanced by gender and race/ethnicity.
Retention was highest in the $10 and $20 enrollment categories (Groups 17–24), ranging from 100% (n=31) for Group 24 ($20 enrollment, $20 retention) to 79% (n=13) for Group 18 ($10 enrollment, $5 retention). Enrollment rates were low and not balanced by gender and race/ethnicity in these groups (Table 2).
The incentive combination that met goals of high and equivalent enrollment across diverse groups, with relative cost effectiveness, was Group 12 ($2 prepaid, $20 retention). This condition achieved 10% (n=53) group enrollment overall and 71% (n=37) retention, with low enrollment ($31) and retention ($65) costs. This combination yielded similar responses by gender and race/ethnicity, between 8% and 12%. Further analysis showed that retention was relatively high and nearly equivalent for men and white/other women (African-American men=86%, white/other men=80%, white/other women=76%), although lower for African-American women (40%).
Multivariate modeling for enrollment showed that retention factors influenced enrollment and that enrollment factors influenced retention; therefore, all factors were included in the models. Significant interactions with gender were identified; therefore, the final models for enrollment were stratified by gender (Table 3). Increased age was associated with increased likelihood of enrollment for men and women (men, p=0.001; women, p=0.030); white/other invitees were more likely to enroll (OR white/other men=1.4; OR white/other women=1.2); and higher retention incentive value produced higher enrollment rates. Significant interactions between men and women were identified for stamped postage (positive for women only); letter version (men responded less to the revised letter); and enrollment incentive (men responded more to $1 and $2 incentives). The $2 bill spurred enrollment in men more than either the $1 or $5 bill (OR men $2=5.5; OR $1=2.6; OR $5=2.1; p=0.001). The Post-it® note did not influence enrollment.
TABLE 3.
Multiple logistic regression models for probability of enrollment and 3-month follow-up, stratified by gender
| Enrollment | 3-month follow-up | ||||
|---|---|---|---|---|---|
| Effect | Level | Men (n=6145) OR (95% CI) | Women (n=6144) OR (95% CI) | Men (n=197) OR (95% CI) | Women (n=334) OR (95% CI) |
| Age (years)a | 21–29 | 1.0 | 1.0 | 1.0 | 1.0 |
| 30–39 | 1.2 (0.7–2.2) | 0.8 (0.6–1.2) | 0.9 (0.3–3.1) | 0.6 (0.2–1.3) | |
| 40–49 | 1.6 (0.9–2.6) | 1.3 (0.9–1.8) | 1.7 (0.5–5.1) | 0.5 (0.2–1.1) | |
| 50–59 | 2.5 (1.5–4.1) | 1.3 (0.9–1.8) | 1.8 (0.6–5.3) | 0.4 (0.2–0.9) | |
| 60–66 | 2.5 (1.3–4.5) | 1.5 (0.9–2.3) | 2.7 (0.7–10.0) | 0.8 (0.3–2.2) | |
| Race: white/other vs African American | 1.4 (1.1–1.9) | 1.2 (1.0–1.5) | 1.1 (0.6–2.1) | 2.7 (1.6–4.4) | |
| Enrollment incentives ($)b | 0 | 1.0 | 1.0 | 1.0 | 1.0 |
| 1 | 2.6 (1.5–4.6) | 1.4 (0.9–1.9) | 0.9 (0.3–2.9) | 1.8 (0.8–4.0) | |
| 2 | 5.6 (3.4–9.3) | 1.9 (1.4–2.7) | 1.1 (0.4–3.2) | 2.3 (1.1–4.8) | |
| 5 | 2.1 (1.0–4.7) | 3.1 (1.6–6.0) | 0.2 (0.0–1.3) | 2.3 (0.6–9.7) | |
| 10 | 1.3 (0.7–2.3) | 1.2 (0.9–1.7) | 1.4 (0.4–5.0) | 4.1 (1.9–9.0) | |
| 20 | 0.6 (0.2–1.8) | 1.5 (0.7–3.3) | 0.9 (0.1–13.1) | 11.5 (1.7–77.9) | |
| Retention incentives ($)b | 0 | 1.0 | 1.0 | 1.0 | 1.0 |
| 5 | 1.1 (0.7–1.7) | 1.3 (0.9–1.8) | 1.9 (0.7–5.0) | 4.2 (1.8–9.5) | |
| 10 | 1.5 (1–2.4.0) | 1.8 (1.3–2.6) | 2.2 (0.9–5.9) | 4.7 (2.1–10.1) | |
| 20 | 1.9 (1.3–3.0) | 2.3 (1.6–3.2) | 5.2 (2.1–13.3) | 3.9 (1.9–8.1) | |
| Post-it® note use | 0.9 (0.7–1.3) | 1.1 (0.9–1.5) | 1.1 (0.5–2.1) | 0.7 (0.4–1.2) | |
| Stamped vs metered postage | 0.8 (0.6–1.1) | 1.4 (1.1–1.8) | 0.9 (0.5–1.6) | 0.7 (0.4–1.1) | |
| Letter Version 2 vs Version 1 | 0.4 (0.2–0.8) | 1.1 (0.7–1.9) | 0.5 (0.1–2.3) | 1.4 (0.5–4.6) | |
Note: Significant interactions with gender were observed for race, enrollment incentive, postage type, and letter version (p=0.002 for each) for the incentive model, and race for the 3-month follow-up model (p=0.039).
The overall age effects were significant for enrollment (p=0.001 for women and p=0.030 for men), but not for follow-up (p>0.2).
Unconditional incentives=$1, $2, $5. Conditional incentives=$10 and $20. All overall incentive effects were significant (p<=0.005), except for the retention model for men (p=0.414).
Men and women were retained equally (men=110/197, 56%; women=187/335, 56%) (Table 3). Retention rates for men were comparable by race/ethnicity (OR=1.1), while white/other women were retained at twice the rate seen for African-American women (OR white/other women=2.6; p<0.001). The enrollment incentive was significantly associated with retention for women (p=0.005) but not for men (p=0.414), although the interaction was not significant (p=0.521). The retention incentive value was important for both men and women (p=0.004 and p<0.001, respectively). Neither postage nor recruitment letter version influenced retention. Comparisons between those retained and those not retained (Table 1) revealed no differences in age, income, or marital status; however, educational attainment was higher among those retained (p=0.026).
Discussion
This study evaluated methods of attracting and motivating enrollment and retention in a web-based health program, within a demographically known large and diverse sample of generally healthy adults. Marketing and survey research strategies, without telephone contact, were examined to identify the most likely combination of mailing and incentive characteristics. Process and cost comparisons were generated so the results could be translated to research and practice within a healthcare system.
Consistent with survey research findings that timing of incentives makes a difference,14 a small prepaid incentive in the recruitment letter proved an effective method of recruitment, reaching rates equal or higher than general mass-mailing recruitment. The prepaid incentive conditions had higher enrollment rates than either no incentive or the incentives promised for task completion. Unlike previous findings,14 the promised enrollment incentives in this study were statistically no better than no incentive for recruitment.
A combination of incentives that maximized cost-effective recruitment and generated balanced enrollment by gender and race/ethnicity and high retention was identified. Enrollment increased with greater prepaid incentive value, even a small unit increase. Monetary novelty contributed differentially to enrollment. A small amount of cash, the novel $2 bill, improved enrollment, attracting men at a higher rate than the $5 incentive. Enrollment rates for the $5 incentive were only slightly higher than those of the $2 condition, although disproportionate by race/ethnicity and gender. The cost per enrollee was generally lower for the $2 condition. As with Jobber26 and Asch,27 using incentives lowered the cost of each enrolled subject as the enrollment rate increased.
Beyond influencing recruitment, declaring the retention incentive amount in the recruitment letter promoted retention. Within the $2 enrollment category, increasing the retention incentive from $10 to $20 improved the retention rate from 56% to 71%, respectively. The highest retention rates followed an earlier experience of receiving a high-value enrollment incentive.
Comparisons of enrollment by gender and race/ethnicity supported selection of the incentive combination that attracted a more diverse enrollment. The novel $2 bill and the promise of $20 for retention achieved the overall 10% recruitment goal, with the retention of nearly 70% of enrollees within roughly equal numbers of men and women. The per-subject recruitment and retention costs ($30 and $65, respectively) were relatively cost effective, considering expenses saved by avoiding telephone contact and the costs of follow-up, data entry, and survey mailing.28
While the study appealed to a diverse population, African Americans and men were less likely to participate in this online diet change program. The enrollment rate for men overall was, however, higher than reports from other food-choice interventions.29
Manipulating the mailing characteristics further revealed the appeal of various strategies. Simpler letter characteristics, including metered postage, encouraged men to enroll, whereas women responded more to postage stamps and the revised letter with its colorful flier. Postage findings refine the work of Dillman,7 who summarized that first-class postage, whether metered or stamp, was usually sufficient for improving mailed survey responses. The type of postage used makes a difference, especially if enrolling men is a priority. The revision of mailed materials to an easy-to-read design to enhance subjects’ interest in working through the stages of opening, reading, and responding21 seemed to work better for enrolling women. Because recipients may discard envelopes that look like “junk mail” before opening them, use of the less costly, normal-size, “business-style” envelope and recruitment letter may have helped increase enrollment.7,21 The personalized Post-it® note failed to increase enrollment.
A weakness of this study is that some eligibility data were not available prior to selecting the recruitment sample, which possibly influenced response rates.28 Nationally, an estimated 70%–75% of families have Internet access at home,30 so eligibility was restricted. Since the online program was in English, people with limited English ability and lower Internet comfort might have abandoned the enrollment process. Lower retention was associated with educational attainment and possibly the noninteractive quality of the online program.31
Strengths included a randomly selected sample with random preassignment, stratified by gender and two race/ethnicity groups, into incentive groups. Sample size was adequate to allow subgroup analyses among the 24 comparison groups. Postal mail and e-mail encouragements to complete the retention survey were consistently applied across incentive groups.
This study provides further understanding of recruitment strategies and the differential responses from women and men and among different race/ethnicity groups for a web-based intervention. Unconditional cash incentives of low monetary value enhanced recruitment. This experience supports the incorporation of survey research and communication science to improve recruitment and retention outcomes.
Acknowledgments
NCI grant U19 CA079689 supported this research, which the authors performed in affiliation with the Cancer Research Network (CRN). The CRN is a collaboration of research programs and enrollee populations of 15 integrated health systems. The overall goal of the CRN is to increase the effectiveness of preventive, curative, and supportive interventions for major cancers among diverse populations and health systems through a program of collaborative research.32
The authors would like to thank for their special contributions Sarah Claud, MPH, and Margie Day from Henry Ford Hospital; Mary Kelley from HealthPartners; Melanie Stopponi, MPA, from Kaiser Permanente-Colorado; Josephine Calvi, MPH, from Kaiser Permanente-Georgia; Linda Shultz, MPH, from Group Health Cooperative; Janine Konkel, MPH, Ed Saunders, MS, Mike Nowak, and Carola Carlier, MSW, from the University of Michigan; Jerianne Heimendinger, ScD, MPH, RD; and Rachel Ballard-Barbash, MD, MPH, from NCI.
No financial disclosures were reported by the authors of this paper.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Griffiths F, Lindenmeyer A, Powell J, et al. Why are health care interventions delivered over the Internet? A systematic review of the published literature. J Med Internet Res. 2006;8(2):e10. doi: 10.2196/jmir.8.2.e10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Strecher V. Internet methods for delivering behavioral and health-related interventions (eHealth) Annu Rev Clin Psychol. 2007;3:53–76. doi: 10.1146/annurev.clinpsy.3.022806.091428. [DOI] [PubMed] [Google Scholar]
- 3.Foy R, Parry J, Duggan A, et al. How evidence based are recruitment strategies to randomized controlled trials in primary care? Experience from seven studies. Fam Pract. 2003;20:83–92. doi: 10.1093/fampra/20.1.83. [DOI] [PubMed] [Google Scholar]
- 4.Koo M, Skinner H. Challenges of Internet recruitment: a case study with disappointing results. J Med Internet Res. 2005;7(1):e6. doi: 10.2196/jmir.7.1.e6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Etter JF, Perneger TV. A comparison of cigarette smokers recruited through the Internet or by mail. Int J Epidemiol. 2001;30:521–5. doi: 10.1093/ije/30.3.521. [DOI] [PubMed] [Google Scholar]
- 6.McClure JB, Greene SM, Wiese C, et al. Interest in an online smoking cessation program and effective recruitment strategies: results from project quit. J Med Internet Res. 2006;8(3):e14. doi: 10.2196/jmir.8.3.e14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Dillman DA. Mail and Internet surveys: the tailored design methods. New York: John Wiley and Sons, Inc; 2000. [Google Scholar]
- 8.Biner PM, Barton DL. Justifying the enclosure of monetary incentives in mail survey cover letters. Psychology and Marketing. 1990;7:153–62. [Google Scholar]
- 9.Doody MM, Sigurdson AS, Kampa D, et al. Randomized trial of financial incentives and delivery method for improving response to a mailed questionnaire. Am J Epidemiol. 2003;157:643–51. doi: 10.1093/aje/kwg033. [DOI] [PubMed] [Google Scholar]
- 10.King KA, Pealer LN, Bernard AL. Increasing response rates to mail questionnaires: a review of inducement strategies. Am J Health Ed. 2001;32:4–15. [Google Scholar]
- 11.Perneger TV, Etter JF, Rougemont A. Randomized trial of use of a monetary incentive and a reminder card to increase the response rate to a mailed health survey. Am J Epidemiol. 1993;138:714–22. doi: 10.1093/oxfordjournals.aje.a116909. [DOI] [PubMed] [Google Scholar]
- 12.Rossoff PM, Werner C, Clipp EC, et al. Response rates to a mailed survey targeting childhood cancer survivors: a comparison of conditional versus unconditional incentives. Cancer Epidemiol Biomarkers Prev. 2005;14:1330–3. doi: 10.1158/1055-9965.EPI-04-0716. [DOI] [PubMed] [Google Scholar]
- 13.Shaw MJ, Beebe TJ, Jensen HL, et al. The use of monetary incentives in a community survey: impact on response rates, data quality, and cost. Health Serv Res. 2001;35:1339–46. [PMC free article] [PubMed] [Google Scholar]
- 14.Church AH. Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opin Q. 1993;57:62–79. [Google Scholar]
- 15.Singer E. The use of incentives to reduce nonresponse in household surveys. In: Groves RM, Dillman DA, Eltinge JL, et al., editors. Survey Nonresponse. New York: John Wiley and Sons, Inc; 2002. [Google Scholar]
- 16.Edwards P, Cooper R, Roberts I, Frost C. Meta-analysis of randomised trials of monetary incentives and response to mailed questionnaires. J Epidemiol Community Health. 2005;59:987–99. doi: 10.1136/jech.2005.034397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Groves RM, Cialdini RB, Couper MP. Understanding the decision to participate in a survey. Public Opin Q. 1992;56(4):475–95. [Google Scholar]
- 18.Groves RM, Presser S, Dipko S. The role of topic interest in survey participation decisions. Public Opin Q. 2004;68(1):2–31. [Google Scholar]
- 19.Groves RM, Singer E, Corning A. Leverage-saliency theory of survey participation: description and an illustration. Public Opin Q. 2000;64(3):299–308. doi: 10.1086/317990. [DOI] [PubMed] [Google Scholar]
- 20.Harrison RA, Holt D, Elton PJ. Do postage-stamps increase response rates to postal surveys? A randomized control trial. Int J Epidemiol. 2002;31:872–4. doi: 10.1093/ije/31.4.872. [DOI] [PubMed] [Google Scholar]
- 21.De Wulf K, Hoekstra JC, Commandeur HR. The opening and reading behavior of business-to-business direct mail. Industrial Marketing Management. 2000;29:133–45. [Google Scholar]
- 22.Helgeson JG, Voss KE, Terpening WD. Determinants of mail-survey response: survey design factors and respondent factors. Psychology & Marketing. 2002;19(3):303–28. [Google Scholar]
- 23.Gendall P. The effect of covering letter personalisation in mail surveys. International Journal of Market Research. 2005;47(4):367–82. [Google Scholar]
- 24.Strecher V, Wang C, Derry H, Wildenhouse K, Johnson C. Tailored interventions for multiple risk behaviors. Health Educ Res. 2002;17(5):619–26. doi: 10.1093/her/17.5.619. [DOI] [PubMed] [Google Scholar]
- 25.Keesling R. Use of web-based data collection to supplement and improve mail survey response rates. Proceedings of the Survey Research Methods Section, American Statistical Association; 2002 Aug 14; New York NY. www.amstat.org/sections/srms/Proceedings/ [Google Scholar]
- 26.Jobber D, Saunders J, Mitchell VW. Prepaid monetary incentive effects on mail survey response. J Bus Res. 2004;57:21–5. [Google Scholar]
- 27.Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997;50:1129–36. doi: 10.1016/s0895-4356(97)00126-1. [DOI] [PubMed] [Google Scholar]
- 28.Couper MP. Web surveys: a review of issues and approaches. Public Opin Q. 2000;64(4):464–94. [PubMed] [Google Scholar]
- 29.Marcus AD, Heimendinger J, Wolfe P, et al. Increasing fruit and vegetable consumption among callers to the CIS: results from a randomized trial. Prev Med. 1998;27(5 Pt 2):S16–S28. doi: 10.1006/pmed.1998.0405. [DOI] [PubMed] [Google Scholar]
- 30.Pew/Internet Latest Trends. Pew Internet & American Life Project. Washington DC: 2007. www.pewinternet.org/trends.asp. [Google Scholar]
- 31.Forsyth I. Teaching and learning materials and the Internet. London, UK: Kogan Page Limited; 2001. [Google Scholar]
- 32.Wagner EH, Greene SM, Hart G, et al. Building a research consortium of large health systems: the Cancer Research Network. J Natl Cancer Inst Monogr. 2005;35:3–11. doi: 10.1093/jncimonographs/lgi032. [DOI] [PubMed] [Google Scholar]