Your inbox probably has more invitations to join online health research surveys than before the COVID-19 pandemic. Online surveys have become an important tool for COVID-19 research when conventional survey methods are not feasible. Yet the response to COVID-19 has also underlined the urgent need for high-quality behavioural data. Is the trend towards online health survey research an indication of practices to come or a stark reminder of the perils of convenient sampling methods? This Comment examines unique opportunities associated with online health research surveys, challenges in implementing and interpreting data from online surveys, and considerations for getting the most out of online health research.
Online surveys provide unique opportunities for research in the COVID-19 era. First, many conventional methods for obtaining behavioural data from people (eg, pencil-and-paper surveys as a part of representative population house surveys) are not feasible during the pandemic. There are few options for collecting real-time information in person as part of an emergency response. Second, although COVID-19 measures might increase the digital divide in accessing health services, policy responses to COVID-19 might decrease the digital divide in terms of completing an online survey.1 Policy responses have elevated broadband access to a fundamental right, providing support for public policies to expand internet access. Third, digital tools and networks (eg, national digital identification numbers) provide increased opportunities for online surveys. Creating and administering an online survey can be done in a fraction of the time and cost needed to organise a similar in-person research study. Fourth, for some sensitive survey items (eg, sexual practices or drug use), people might prefer an online survey compared with one administered in person.2 With the backdrop of COVID-19, these same behaviours might change over time, increasing the importance of this research agenda.
However, there are also persistent threats to the scientific rigour of online health research studies related to study design and implementation. In terms of study design, problems can arise related to sampling that could influence the generalisability of findings. Although the number and diversity of active internet users grows worldwide, there is still a risk of selection bias,3 and people without internet access or who are unwilling to participate in online surveys might still not be reached.4 During survey implementation, the absence of human interaction can encourage hesitation and make it difficult to differentiate junk mail from research.5 Distrust of research might also affect the likelihood of vulnerable groups joining an online survey. Our International Sexual Health and Reproductive Health (I-SHARE) project has coordinated more than 30 separate online surveys on sexual and reproductive health.6 Considerations of study design, implementation, and data analysis used in the I-SHARE project and that can improve the rigour of online surveys are presented in the figure .
Figure.
Ensuring rigour of online surveys
Study design considerations related to probability panels and real-time assessment can reduce bias and increase the rigour of online survey findings. For example, mobile-phone panels, nationally representative online panels, and lists of residents with a digital identification number can provide samples in many countries.7 Although digital panels are now rare and confined to high-income countries, mobile-phone panels are sampling options in a broader number of settings. Additionally, real-time assessment of recruitment efforts can ensure specific subgroups are included. For example, placing online and offline advertisements (eg, print materials at social gatherings) can increase online survey participation among men, ethnic or racial minorities, and other groups.8 Additionally, online health research surveys can still use population-representative sampling frames. Several population-representative cohorts have organised follow-up online surveys among a subset of individuals during the COVID-19 pandemic.9
Survey implementation measures, such as creating partnerships with local organisations, can increase interaction and the digital visibility of the study, and establish trust in online research studies. For example, the I-SHARE survey leveraged national (family planning and academic research organisations, and non-profit advocacy groups) and global (United Nations Family Planning Association, WHO) relationships with organisations focused on sexual and reproductive health.
Finally, prespecified data analysis plans and some analytical methods can enhance rigour. Prespecified analysis plans can be formalised in a publication or data repository, increasing transparency in research findings.10 In addition, propensity score matching (in cases where a population sample exists) might decrease coverage error and make online samples more closely approximate population-representative samples (appendix).
These and related considerations described in the appendix provide a framework for increasing the rigour of online surveys. Further research is needed, particularly research related to ensuring that ethnic and racial minorities are appropriately sampled. Online surveys were increasing before COVID-19, and COVID-19 measures are only accelerating this trend—the invitations to complete an online survey will probably continue. Now, it is important to make the most of the online survey data available.
Acknowledgments
We would like to thank Sarah Van de Velde, Maximiliane Uhlich, Dan Wu, Stefano Eleuteri, Wei-Hong Zhang, and Adesola Olumide for their comments on an earlier version of the manuscript. JDT received financial support from the National Institute of Allergy and Infectious Diseases (K24AI143471). We declare no competing interests.
Supplementary Material
References
- 1.Watts G. COVID-19 and the digital divide in the UK. Lancet Digit Health. 2020;2:e395–e396. doi: 10.1016/S2589-7500(20)30169-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gnambs T, Kaspar K. Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis. Behav Res Methods. 2015;47:1237–1259. doi: 10.3758/s13428-014-0533-4. [DOI] [PubMed] [Google Scholar]
- 3.Eysenbach G, Wyatt J. Using the Internet for surveys and health research. J Med Internet Res. 2002;4:E13. doi: 10.2196/jmir.4.2.e13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Ball HL. Conducting online surveys. J Hum Lact. 2019;35:413–417. doi: 10.1177/0890334419848734. [DOI] [PubMed] [Google Scholar]
- 5.Evans Joel R, Mathur A. The value of online surveys. Internet Res. 2005;15:195–219. [Google Scholar]
- 6.Michielsen K, Larrson EC, Kågesten A, et al. International sexual health and REproductive health (I-SHARE) survey during COVID-19: study protocol for online national surveys and global comparative analyses. Sex Transm Infect. 2020 doi: 10.1136/sextrans-2020-054664. published online Oct 20. [DOI] [PubMed] [Google Scholar]
- 7.Dabalen A, Etang A, Hoogeveen J, Mushi E, Schipper Y, von Engelhardt J. Mobile phone panel surveys in developing countries: a practical guide for microdata collection. 2016. https://openknowledge.worldbank.org/bitstream/handle/10986/24595/9781464809040.pdf
- 8.Ali SH, Foreman J, Capasso A, Jones AM, Tozan Y, DiClemente RJ. Social media as a recruitment platform for a nationwide online survey of COVID-19 knowledge, beliefs, and practices in the United States: methodology and feasibility analysis. BMC Med Res Methodol. 2020;20:116. doi: 10.1186/s12874-020-01011-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Himelein K, Eckman S, Lau C, Mckenzie D. Mobile phone surveys for understanding COVID-19 impacts: part I sampling and mode. World Bank. April 7, 2020 https://blogs.worldbank.org/impactevaluations/mobile-phone-surveys-understanding-covid-19-impacts-part-i-sampling-and-mode [Google Scholar]
- 10.Thomas L, Peterson ED. The value of statistical analysis plans in observational research: defining high-quality research from the start. JAMA. 2012;308:773–774. doi: 10.1001/jama.2012.9502. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.