Abstract
Objectives
The objectives of the study were 1) to assess the effect of cash lottery on participation rates in a web-based study of physical activity and joint health and 2) to compare recruitment via direct e-mail versus advertisement in an online newsletter.
Methods
A sample of 1,150 individuals, randomly selected from a database of members of the Canadian Association of Retired Persons (CARP), was e-mailed a request to participate in an online survey, with follow-up e-mails after 1 and 2 weeks. The sample was randomly split into two groups. Half the sample was offered entry into a cash draw with a $500 grand prize and five $100 prizes, whereas the other half was not offered any incentive. In addition, a brief advertisement about the survey (without an incentive) was placed in an online newsletter that was circulated to 14,000 randomly selected CARP members.
Results
In the incentive group, 305 (53.0%) clicked on the hyperlink and visited the website and 84 (14.6%) completed the survey. In the group who received no incentive, 280 (48.7%) clicked on the link and 59 (10.3%) completed the survey. Of those who received the online newsletter, 492 (3.5%) visited the website and 106 (0.76%) completed the survey.
Conclusion
A relatively modest financial incentive in the form of a cash lottery significantly increased participation rates in an online health survey. Recruitment through a newsletter advertisement had a very low yield compared to direct e-mail.
Key words: Internet, survey, response rates, incentives
Résumé
Objectifs
1) Analyser l’effet d’un tirage de prix en argent comptant sur les taux de participation à une étude en ligne sur l’activité physique et la santé articulaire et 2) comparer le recrutement direct par courriel à la publication d’une annonce dans un bulletin en ligne.
Méthode
Nous avons demandé par courriel à un échantillon de 1150 personnes, sélectionnées au hasard dans la base de données des membres de l’Association canadienne des individus retraités (ACIR), de participer à un sondage en ligne, et nous leur avons envoyé des messages de suivi après 1 et 2 semaines. L’échantillon a été divisé en deux groupes de façon aléatoire. La moitié de l’échantillon s’est fait proposer de participer à un tirage de prix en argent comptant (un grand prix de 500 $ et cinq prix de 100 $), tandis que l’autre moitié n’a reçu aucune incitation. Par ailleurs, une brève annonce à propos du sondage (sans mesure incitative) a été publiée dans un bulletin en ligne envoyé à 14 000 membres de l’ACIR sélectionnés au hasard.
Résultats
Dans le groupe ayant reçu une incitation, 305 personnes (53 %) ont cliqué sur l’hyperlien et visité le site Web, et 84 (14,6 %) ont répondu au sondage. Dans le groupe n’ayant pas reçu d’incitation, 280 personnes (48,7 %) ont cliqué sur le lien et 59 (10,3 %) ont répondu au sondage. Parmi les personnes ayant reçu le bulletin en ligne, 492 (3,5 %) ont visité le site Web et 106 (0,76 %) ont répondu au sondage.
Conclusion
Une incitation financière relativement mineure — le tirage de prix en argent comptant — a significativement accru le taux de participation à un sondage en ligne sur la santé. Le recrutement au moyen d’une annonce dans un bulletin a donné des résultats beaucoup moins bons que l’envoi direct de courriels.
Mots clés: Internet, questionnaires, taux de réponse, mesures incitatives
Footnotes
Source of Support: The study was funded by a grant from the Canadian Institutes of Health Research. Dr. Kopec was supported by a Michael Smith Foundation for Health Research Senior Scholar Award.
Conflict of Interest: None to declare.
References
- 1.Dillman DA, Smith JD, Christian LM. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley; 2009. [Google Scholar]
- 2.Church AH. Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opin Q. 1993;57(1):62–79. doi: 10.1086/269355. [DOI] [Google Scholar]
- 3.James JM, Bolstein R. Large monetary incentives and their effect on mail survey response rates. Public Opin Q. 1992;56(4):442–53. doi: 10.1086/269336. [DOI] [Google Scholar]
- 4.Warriner K, Goyder J, Gjertsen H, Hohner P, McSpurren K. Charities, No; Lotteries, No; Cash, Yes: Main Effects and Interactions in a Canadian Incentives Experiment. Public Opin Q. 1996;604:542. doi: 10.1086/297772. [DOI] [Google Scholar]
- 5.Spry VM, Hovell MF, Sallis JG, Hofsteter CR, Elder JP, Molgaard CA. Recruiting survey respondents to mailed surveys: Controlled trials of incentives and prompts. Am J Epidemiol. 1989;130(1):166–72. doi: 10.1093/oxfordjournals.aje.a115309. [DOI] [PubMed] [Google Scholar]
- 6.Marrett LD, Kreiger N, Dodds L, Hilditch S. The effect on response rates of offering a small incentive with a mailed questionnaire. Ann Epidemiol. 1992;2(5):745–53. doi: 10.1016/1047-2797(92)90019-M. [DOI] [PubMed] [Google Scholar]
- 7.Kalantar JS, Talley NJ. The effects of lottery incentive and length of questionnaire on health survey response rates: A randomized study. J Clin Epidemiol. 1999;52(11):1117–22. doi: 10.1016/S0895-4356(99)00051-7. [DOI] [PubMed] [Google Scholar]
- 8.Whiteman MK, Langenberg P, Kjerulff K, McCarter R, Flaws JA. A randomized trial of incentives to improve response rates to a mailed women’s health questionnaire. J Womens Health (Larchmt) 2003;12(8):821–28. doi: 10.1089/154099903322447783. [DOI] [PubMed] [Google Scholar]
- 9.Roberts LM, Wilson S, Roalfe A, Bridge P. A randomised controlled trial to determine the effect on response of including a lottery incentive in health surveys. BMC Health Serv Res. 2004;41:30. doi: 10.1186/1472-6963-4-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Tamayo-Sarver JH, Baker DW. Comparison of responses to a US 2 dollar bill versus a chance to win 250 US dollars in a mail survey of emergency physicians. Acad Emerg Med. 2004;11(8):888–91. doi: 10.1197/j.aem.2004.01.010. [DOI] [PubMed] [Google Scholar]
- 11.Göritz AS. Incentives in Web-based studies: Methodological issues and a review. Int J Internet Sci. 2006;1(1):58–70. [Google Scholar]
- 12.Brennan M, Rae N, Parackal M. Survey based experimental research via the web: Some observations. Marketing Bull. 1999;10:57–65. [Google Scholar]
- 13.Cobanoglu C, Cobanoglu N. The effect of incentives in web surveys: Application and ethical considerations. Int J Market Res. 2003;45(4):475–88. doi: 10.1177/147078530304500406. [DOI] [Google Scholar]
- 14.Porter S, Whitcomb ME. The impact of lottery incentive on student survey response rates. Res Higher Educ. 2003;44(4):389–407. doi: 10.1023/A:1024263031800. [DOI] [Google Scholar]
- 15.Göritz AS. Cash lotteries as incentives in online panels. Soc Sci Comput Rev. 2006;24(4):445–59. doi: 10.1177/0894439305286127. [DOI] [Google Scholar]
- 16.Göritz AS. Lotteries as incentives in longitudinal web studies. Soc Sci Comput Rev. 2007;25(1):99–110. doi: 10.1177/0894439306292268. [DOI] [Google Scholar]
- 17.Bosnjak M, Tuten TL. Prepaid and promised incentives in web surveys: Experiment. Soc Sci Comput Rev. 2003;21(2):208–17. doi: 10.1177/0894439303021002006. [DOI] [Google Scholar]
- 18.J Med Internet Res. 2004.
- 19.Akl EA, Maroun N, Klocke RA, Montori V, Schunemann HJ. Electronic mail was not better than postal mail for surveying residents and faculty. J Clin Epidemiol. 2005;58(4):425–29. doi: 10.1016/j.jclinepi.2004.10.006. [DOI] [PubMed] [Google Scholar]
- 20.Mehta R, Sivadas E. Comparing response rates and response content in mail versus electronic mail surveys. J Mark Res Soc. 1995;37:429–39. [Google Scholar]
- 21.Koo M, Skinner H. Challenges of internet recruitment: A case study with disappointing results. J Med Internet Res. 2005;7(1):e6. doi: 10.2196/jmir.7.1.e6. [DOI] [PMC free article] [PubMed] [Google Scholar]