Survey-based observational research: Quality, relevance and optimal use in the post-COVID-19 era

The coronavirus disease 19 (COVID-19) pandemic has led to a reconsideration of many activities hitherto considered routine, including those related to medical education and research. Face-to-face academic meetings have been largely replaced or supplemented with online or hybrid meetings. The use of social media channels such as Twitter for academic knowledge sharing has increased exponentially during this time.¹ The cessation of traditional research activities directly involving patients resulted in researchers directing their research activities online, particularly in the early days of the pandemic. Consequently, survey-based online observational research has rapidly proliferated. In this context, we in this editorial discuss how to make optimal use of survey-based research in the post-COVID-19 era.

Types of survey-based research

Surveys could enable the understanding of perspectives of medical practitioners regarding controversial topics. As an example, when the COVID-19 pandemic was just taking a foothold, there was considerable uncertainty regarding the use of various immunosuppressive drugs for fear of exacerbating the severity of COVID-19. Understanding the perspectives of rheumatologists who commonly use such drugs was useful to allay uncertainties regarding the use of these drugs.² Similarly, one of the drugs touted as a miracle cure for COVID-19 was hydroxychloroquine (HCQ). Since HCQ is used by a majority of patients with systemic lupus erythematosus, a survey of these patients was useful to provide preliminary information regarding safety or effectiveness of this drug in the background of COVID-19.³

Surveys could also enable consensus development among experts internationally using the Delphi method or other similar methods without requiring face-to-face meetings (which not only are costly but might be inconvenient currently). Examples of this include attempts to develop consensus-based definitions for predatory journals.⁴ Similarly, online surveys could be useful to obtain the perspectives of an international pool of experts regarding specific practice questions. As an example, a recent survey attempted to understand the perspectives of international experts in Takayasu arteritis (a rare disease) on the use of the imaging modality of positron emission tomography to homogenise recruitment of patients into clinical trials of Takayasu arteritis.⁵

Surveys could also be useful to gather data to enable the development of policies related to education or public health. As an example, a recent survey attempted to understand the perspectives of physicians from India regarding their perceived adequacy of rheumatology training during undergraduate medical education or postgraduate training.⁶ It was envisaged that deficiencies thus identified would support the advocacy for relevant curricular reforms.

Points to consider regarding the quality of survey-based observational research

The Checklist of Reporting for Internet E-Surveys provides a framework to maintain the quality of surveys.⁷ A few important considerations have been discussed below.

The sampling strategy should be carefully considered to enhance the representativeness of a survey. Often, online surveys are disseminated through social media channels without an attempt to include specific target populations. Therefore, the denominator (i.e. people who could have participated but chose not to) is uncertain. There remains a possibility that such surveys are unrepresentative as they reflect the opinion of those who are active on social media channels alone, unless other sampling strategies are used to enhance the outreach to a wider population (conceptually similar to the healthy user bias wherein healthier participants are more likely to participate in studies, thereby biasing the said studies to observe more favourable outcomes than in real life).⁸ The survey sample is well defined when experts are individually invited to participate in a survey.⁵ In any case, the sampling strategy along with its advantages or disadvantages should be clearly mentioned in the survey report.

It is not possible to obtain written informed consent for online surveys. However, the introductory part of the survey should provide an invitation to participate in the survey, seeking permission of those being surveyed for participation as well as for retaining some anonymised demographic information such as age and gender. A balance is required between anonymisation and retention of basic demographic information such as age and gender to enable more effective appraisal of the survey results. Participants might be asked to opt-in to have their email addresses available to survey investigators for further questions (e.g. when answers are incomplete or unclear). Many countries provide exemption from ethical review for survey-based research; however, relevant local regulations should be consulted during the planning of the survey.

The questionnaires used for survey-based studies should be developed with adequate rigor. It should be carefully considered how long the surveys take to fill out. Surveys that are too long risk being incompletely filled. The survey questionnaires should be validated during their development to establish face, content and construct validity. Test–retest and inter-rater reliability should be evaluated. The questionnaires should be piloted to a few before wider dissemination to iron out any flaws based on feedback obtained from participants. The use of adaptive questions may avoid redundant questions based on their earlier choices, thereby maintaining more effective engagement of participants.^7,9

Online surveys might be filled multiple times by a single participant. The use of techniques such as screening of internet protocol addresses or manually screening responses to demographic questions (when the survey samples are of a smaller size) might enable the identification and deletion of duplicate entries.¹⁰Table 1 summarises the major considerations relevant for the quality of survey-based observational research.

Table 1.

Relevant quality considerations for survey-based observational research.

Defining sample of the target population (i.e. participants)

Research ethics committee approval and informed consent

Validation of the questionnaire (face, construct and content validity)

Reliability of survey questionnaires (test–retest, inter-rater reliability)

A priori consideration of statistical tools and analyses of results

Piloting of survey

Clearly defined strategies to weed out duplicate entries

Robust reporting

Conclusions

A clearly defined research question which can be answered by the survey is an essential consideration for high-quality surveys. The investigators must also critically analyse the need for the survey, viz., does the survey really answer a question relevant for patient care or better understanding of a given scenario. Ensuring the representativeness of the survey sample, validation and piloting of surveys before wider dissemination are other important considerations. Keeping these points in mind might enable investigators to improve the quality of their surveys as well as help reviewers and editors appropriately evaluate the quality of surveys submitted for publication.