Conducting Qualitative Research Online: Challenges and Solutions

Abstract

What ways of thinking and concrete strategies can assist qualitative health researchers to transition their research practice to online environments? We propose that researchers should foreground inclusion when designing online qualitative research, and suggest ethical, technological and social adaptations required to move data collection online. Existing research shows that this move can aid in meeting recruitment targets, but can also reduce the richness of the data generated, as well as how much participants enjoy participating, and the ability to achieve consensus in groups. Mindful and consultative choices are required to prevent these problems. To adapt to ethical challenges, researchers should especially consider participant privacy, and ways to build rapport and show appropriate care for participants, including protocols for dealing with distress or disengagement, managing data, and supporting consent. To adapt to technological challenges, research plans should choose between online modalities and platforms based on a clear understanding of their particular affordances and the implications of these. Finally, successful research in virtual social environments requires new protocols for engagement before data collection, attention to group numbers and dynamics, altered moderator teams and roles, and new logistical tasks for researchers. The increasing centrality of online environments to everyday life is driving traditional qualitative research methods to online environments and generating new qualitative research methods that respond to the particularities of online worlds. With strong design principles and attention to ethical, technical and social challenges, online methods can make a significant contribution to qualitative research in health.

Key Points for Decision Makers

Qualitative research can thrive in online modalities if supported by sound methodology and carefully adapted methods.

In moving to online data collection, equity must be a central consideration; online modalities may increase opportunities to participate for some and exclude others.

Different technological platforms offer different strengths; adaptation is required to manage the virtual social environment and address particular ethical challenges in online engagement.

Interviewer: Now, you were just about to say something when you froze.

Participant: Yeah …

Interviewer: Oh, now you’re freezing again.

Participant: Let me just close this other …

Interviewer: No, I’ve got you again, that’s, you’re coming back.

Participant: Ok, good, I just closed a window I had open.

Interviewer: Just give me one second and I’ll just shout upstairs at my daughter who is probably watching something.

Participant: Ok.

Interviewer: (has conversation with daughter) Sorry about that.

Participant: That’s ok. It’s part of, part of the world we live in.

Interviewer: It is. The cat’s been trying to come and have a look at you as well, but I’ve managed to keep her down.

Excerpt from qualitative interview conducted on a videoconferencing platform in 2020

Many readers will recognise the encounter above, and may have had interactions like it, attempting to balance the personal and the professional, attempting to transpose rules and norms of one milieu into another, attempting to connect against distraction and technological difficulties. These issues are perhaps more acute for research interactions—like the one above—than for everyday interactions. In research, the need to generate meaningful findings, the requirements of human research ethics, and limits of time and resources increase the stakes. The challenge is arguably greater still for qualitative research, where participants are asked to speak in depth about often very personal, private or challenging issues, and rapport and support for participants can be critical to success. Our aim here is to provide practical assistance to help qualitative researchers and participants succeed in this online terrain.

Qualitative methods are a natural fit for patient-centred outcomes and health preferences research, as they allow the study of participants’ experiences, choices and actions from the participant’s perspective. While qualitative methods are often used as a preliminary step in the development of quantitative instruments or studies [1], qualitative studies provide complex and patient-centred insights in their own right [2], and are now commonly synthesised to inform health policy, health services, and health technology assessment [3]. Qualitative health researchers are increasingly turning to online platforms to collect data, whether in response to social distancing requirements during the COVID-19 pandemic [4], to research online worlds as unique cultures and communication environments [5], or because innovative methods can achieve novel aims [6]. Moving research online is not a simple ‘like-for-like’ transfer however; the transition can be a disorienting struggle even for experienced researchers.

Qualitative research is diverse and heterogeneous, with different underpinning assumptions, aims, methods for data collection and analysis, and reporting styles [7]. We will concentrate only on interview and focus group methods because they are frequently used in patient preferences research. The online environment is reinventing these methods, with adaptations including online focus groups, email interviewing, Instant Messaging (IM) interviewing, and the use of internet-based video interviewing [8]. There are many other qualitative methods that can be used in the online environment, including netnography [9], online visual research methods [10], and social media research methods [5], but these are beyond the scope of this paper.

Qualitative researchers have adapted repeatedly to technological change, both in the mode of engagement with participants, and the collection, transformation and storage of data. A longitudinal view reveals multiple moments of technological recalibration for qualitative researchers. For some time, researchers accustomed to face-to-face interviews asked whether telephone interviews were acceptable, but they are now both commonplace and recognised as highly suitable for interaction with certain participants, e.g. with elites [11, 12]. As natural language processing improves and data storage and processing speed increases, human transcribers are being replaced with automated transcription software, and transcripts with clipping and coding digital recordings directly [13]. These changes have not been linear—technologies are reinvented and recombined over time—but change and technological adaptation have been a constant. In each of these transformations, new issues arise that need to be considered.

The authors are experienced qualitative researchers who share an interest in methodology, methods and research ethics. This paper emerged through discussion of issues that had arisen in our online experience to date and potential issues we could foresee given the different topics and specific populations we research, along with looking to the literature for answers to questions we faced in our practice. We are writing in early 2021, when social distancing requirements in many countries have greatly accelerated a nascent move towards greater online data collection. As the qualitative research community continues to come to terms with these changes, we consider the opportunities and challenges of online data collection that pandemic conditions have made evident.

Doing Qualitative Research in a Virtual Environment: Opportunities, Challenges and Solutions

A recent scoping review compared face-to-face with online research studies of health and illness experiences. The authors concluded that while online methods appear to increase the likelihood of obtaining the desired sample, responses are typically shorter, less contextual information is obtained, and relational satisfaction and consensus development are lower [14]. This does not mean that online methods are inferior, but it does mean that researchers should deliberately plan to mitigate their potential weaknesses.

In the following sections, we consider a set of interconnected issues, taking a lead from Davies and colleagues’ scoping review [14]. First, we will argue that while the online environment may facilitate participation, the move online can enable or hinder inclusion. We will then consider the ethical, technological and social adaptations required in online data collection to, among other things, maximise data quality and care for participants. We note as a background premise that usual qualitative study design considerations—the need for sound aims, research questions, recruitment and sampling strategies, interview or focus group guides and analysis strategies—still hold. We will focus on adaptation of procedures, with sound research design principles assumed [15, 16].

Moving Online Can Enable or Hinder Inclusion

Unjustly excluding people because of their technological or material circumstances is an old research ethics problem that potentially takes a new form in online research, potentially altering the accessibility of research for participants in positive or negative ways. Transitioning from face-to-face to online data collection can broaden access by lifting geographic limits. Online data collection can reduce the burdens of time and cost of participating in research. Participants do not have to travel or host a researcher, and it may be more convenient to conduct interviews and focus groups outside of working hours. These adjustments are likely to make participation easier or more appealing for some groups that previously faced practical limitations to taking part in qualitative research. For example, people with limited mobility, as well as caregivers, may find online participation from home inviting because they do not need to make the same sorts of accommodations that can stand in the way of in-person research [17].

Conversely, online data collection may also limit participation only to those who have a web-enabled device, and sometimes authority to install software. Online video platforms require a good-quality internet connection and relatively high data usage. People without access to fast and reliable internet, as well as people with limited access to data, may find it difficult or less appealing to participate. Online data collection risks excluding, or creating additional burdens and considerable stress for, participants who do not feel competent in the use of technology. Finally, not all technology can accommodate the needs of participants living with specific disabilities.

Researchers can mitigate these barriers to participation and inclusion through mindful and consultative technological and logistical choices. For those with limited access to technology, video conferencing platforms may be inappropriate; inclusion may require conducting an interview without video (audio only) or via telephone to reduce the need for a high-quality internet connection. Researchers may also consider methods such as email interviewing or IM interviewing, which offer accessibility benefits (e.g. more time for participant reflection, less data-intensive technology) but also disadvantages (e.g. requires sufficient literacy) [8]. Researchers can provide participants with data credit vouchers so that they can participate in video calls without the burden of additional data costs. Different platforms offer different participation options for people with disabilities (Table 1), and accessibility options are improving. Accessibility experts and advocacy groups are a good source of information (e.g. [18, 19]). As in face-to-face data collection, specialist advice, including from participants themselves, can assist inclusion of people who use augmentative or alternative communication devices. Researchers should also be flexible with, and take the lead from, participants to maximise inclusion, as participants may have identified or developed solutions that make video conferencing platforms more accessible for them. People with impaired hearing, for example, may find it difficult to rely on lip-reading in video calls, but could participate via a synchronous text chat interview, or on a video platform with the right speech-to-text captioning tool, or with a sign language interpreter pinned next to the main speaker on screen [20-22].

Traditionally, meeting in person has helped shape sampling and recruitment strategies for studies. The location of the research team has often determined the geographic parameters of the study population because face-to-face interviews and focus groups have been the norm for data collection. Online platforms potentially eradicate some geographic barriers and may prompt researchers to think differently about their research questions. While it may be tempting to substantially widen sampling and recruitment because online methods have made it possible, researchers should remain mindful of the importance of methodological concerns. Study populations are shaped by considerations other than practicality. Researchers must be clear about why they have identified the population of interest and how that sample will help them answer their study questions. It may be that geographic location or experience of a particular healthcare system remains an important factor to capture.

Practical Ways to Adapt to Technological, Social and Ethical Challenges in Online Research

Successful online data collection requires three kinds of adaptation: to ethical challenges, to a new technological environment, and to a new social environment. These are interconnected but for clarity we deal with each of them in turn below.

Adapting to Ethical Challenges

In addition to usual research ethics considerations, online data collection raises special challenges. For example, online data collection creates different privacy risks. Online engagement with video means a researcher (and if a focus group, other group members) can potentially see and hear a participant’s domestic space. There are other privacy considerations—some communication platforms require a participant profile, including name, date of birth, email address and/or mobile phone number; participants may not want a profile, or if they have one they may not want to disclose it. Supporting people to participate anonymously may be vital for some populations/research topics. Participants also need access to a quiet and private space. For example, participants who rely on public libraries for internet access are unlikely to be able to do this with privacy.

During in-person research, we use ordinary actions to show our presence and care, or to create rapport: small talk, sharing a beverage, handing a tissue to a distressed participant, closing an encounter by walking a participant out of the building. Online data collection means the loss of this embodied care. Researchers need to develop strategies to establish rapport or comfort a distressed participant; these protocols should be included in ethics applications. We suggest the following adaptations to address these and other important ethical concerns.

Develop a protocol for dealing with distress or disengagement Common in research with vulnerable participants or on sensitive topics, we recommend these protocols for all online qualitative research. Develop clear strategies for how you will deal with an interview participant who becomes visibly distressed or unresponsive, moves away from the screen, shuts down the platform, does not return from an agreed comfort break, or where you witness problematic interactions with other people in the participant’s setting. A similar protocol is advisable for focus groups to deal with distress, or with abuse or discriminatory actions between participants. Ensure you have an alternative means to contact each participant and let participants know in advance under what circumstances you will contact them via this alternative channel.

Ensure video and/or audio recordings are stored appropriately Researchers should check where an online platform is storing recordings and their privacy policy. Using a platform’s cloud service can be in contravention of local privacy legislation (e.g. the European Union’s General Data Protection Regulation [GDRP]) or ethical approval; choose a platform that allows researchers to store recordings on their computer or institutional cloud service. For sensitive research topics, recording via an offline audio device (e.g. digital recorder) provides greater security.

Decide how consent will be recorded Consent processes can be less straightforward for online research; several methods are available, each with benefits and disadvantages. Asking participants to return a written consent form prior to data collection can place burdens on participants and requires a printer and scanner/smartphone. Online platforms (e.g. DocuSign) can be efficient but raise participant access, competency and data security concerns. Adobe Acrobat offers several methods including allowing participants to ‘sign’ via a smartphone screen, print and scan. Researchers can seek and record verbal consent (if acceptable to their ethics review board); this may be preferable, both for its lower burden on participants and to encourage the participant to ask questions before participating. Consider doing this in an introductory interaction (before the data collection event), especially for focus groups; this allows more attention to individual questions, and greater confidentiality. Flexibility is important as methods should suit participants’ comfort and capabilities.

Address online data collection challenges in ethics applications Ethics review boards will vary in their understanding of and tolerance for online data collection. As with face-to-face research, anticipate and address concerns: provide a logic for your study design, explain how the chosen data collection method(s) and platform meet the needs of the participants and the research topic. Be transparent about challenges and outline specific strategies for enhancing participation and offsetting risk. If your online research engages participants in new and unfamiliar locations, researchers should investigate whether their local ethics board approval will be sufficient to work in that context. Seeking advice from ethics review boards in advance can reveal common concerns and offer solutions.

Adapting to Technological Challenges: Hardware and Software

Planning ahead As online research events rely on the functionality and management of technology, both hardware and software, technological logistics should be central to research planning. Before commencing data collection, researchers should ensure that prospective participants have (1) access to hardware (e.g. phone, tablet, computer); (2) a reliable internet connection; (3) familiarity with the chosen platform; and (4) adequate support to respond to technological problems. Participants may need technical coaching and support before data collection occurs.

Affordances that facilitate desired social interactions Different online communication platforms have different affordances [4], and these functionalities enable, for example, different degrees of interactivity, data recording, confidentiality and privacy, and security (Box 1). Although ideally platforms would be chosen to suit the participants, in some instances a researcher’s institution, or local legislation will dictate the use of certain platforms for reasons including licensing or security. Issues to consider in selecting and managing the technological aspects of online research include the following.

• Microphone and camera control: allows either, or both, participant or host to manually control their own or others’ cameras and microphones, helpful for managing background noise or speaking order if required.

• Chat functions: allows short textual comments or questions to be posted by participants, usually in a sidebar from the main screen, and usually without disrupting the verbal conversation.

• Breakout rooms: small subgroup discussions that can be separated out from the main meeting; host/s can join in and out, for example to answer or ask questions, or to facilitate discussions. Some platforms can automatically assign participants into rooms, with a mandatory timed finish, and automatically rejoin participants back into the main meeting.

• Participant polling: short surveys or votes to gauge participant sentiments or show preferences.

• Screen sharing: allows any participant to share the contents of their own screen, which is useful for sharing digital images or other materials the participant might want to introduce to the discussion.

• Screen annotation: interactive screen-based textual and drawing tools, enabling participants to visually mark the content shown on screen.

• Live subtitles and captioning: an additional service, often requiring subscription, that enables live subtitling of video calls, using a ‘speech to text’ recognition software. This may aid the participation of people living with hearing impairment [19].

Anonymity of participants If anonymity of participants is important, choose a platform that can easily control username displays and prepare participants to control how they present themselves. Some platforms display both first and surnames by default when entering an online meeting, therefore ensure participants know how to edit their display name. Avoid online platforms that require an account sign-up and automatically displays the user’s account name or contact phone number, as this compromises privacy and confidentiality. As participants may join the virtual research from their own homes or private offices, pre-research coaching should include the option of using virtual backgrounds for greater privacy protection.

Recording, screenshots and transcription Certain platforms offer recording of online interactions and transcription of audio data. Be sure to check how and where these data files will be stored and secured (see ‘Adapting to Ethical Challenges’ section). A screenshot allows anyone accessing the online event to take a photograph of the screen. This can be a useful tool in research but also allows participants to take recordings and screenshots without the knowledge of researchers and others. Consent for recording should be discussed with everyone taking part prior to commencing any online data collection activities, recording turned off for participants, and participants instructed not to make their own offline recordings.

Manually controlled or password entry Controlled entry by the host usually comes in the form of a ‘waiting room’, whereby the host manually admits participants. This gives hosts a greater degree of control but will also require more time and attention, particularly for larger groups. Password entry allows anyone with a password to the meeting to enter automatically and may save more time. Many research institutions and Human Research Ethics Committees (HRECs) already require password protection for online research.

Box 1 Platform functions checklist.

When choosing an appropriate platform, check these specific technological affordances against the needs and suitability for your research method and participants:

For managing privacy, confidentiality and security of the participants and the research space:

• ✓ Password entry

• ✓ Admission and removal of participants

• ✓ Username display control

• ✓ Virtual background

For facilitating effective social interactions online

• ✓ Microphone and camera control

• ✓ Chat functions

• ✓ Breakout rooms

• ✓ Participant polling

• ✓ Screen sharing

• ✓ Screen annotation

For managing data collection and storage

• ✓ Built-in video and audio recording

• ✓ Subtitles and captioning

• ✓ Secure storage of recorded data

• ✓ Screenshot

Adapting to Social Difference: Knowing the Virtual Social Environment and Working with It

Compared with face-to-face research settings, researchers will have less control over potential interruptions to online data collection activities, as they cannot be physically present to offer alternative arrangements or interventions. Some participants may be practiced in online interactions as part of their daily work or social routine, while others will not [23]. Being prepared to manage interruptions, unpredictability and diversity of comfort level with online interactions is crucial. Below we suggest some adaptations to manage the social dimensions of online research.

Pre-research briefing/check-in Conducting a pre-research briefing can help participants be informed about what to expect and ensure they are comfortable using the online technologies and platforms. If you are working with participants who are vulnerable, have challenges in communicating, or are not familiar with using online technologies, supporting their communication and technology-use needs before data collection is crucial. This can also help build rapport to enhance participants’ relational satisfaction with participation.

Determining numbers in a focus group Compared with face-to-face research, online group interactions demand more cognitive effort for both moderators and participants [23]. Online interactions can also have a slower flow due to minor lags in screen interactions, which tends to exacerbate as the number of participants increase. Maximum numbers will likely be smaller than in face-to-face interactions; we recommend four to six participants for online focus groups. The goal is to not only ensure enough ‘energy’ in the room to sustain interaction but to also make facilitation manageable and the experience more enjoyable for participants.

Manage the energy in the ‘room’ Online focus groups and interviews require more than facilitating the content and flow of the discussion. Focused social interactions between people on a research topic, particularly with unfamiliar others, are particularly mentally demanding. Ways to manage this include slowing down the speed of the conversation with slightly longer pauses between sentences or questions and taking shorter breaks more frequently if a focus group runs for more than an hour. In our experience, online group modalities can encourage participants to take discrete turns rather than interacting in a dynamic flow; this may be offset to some extent by smaller group size and less intrusive moderation that creates more space for participant talk.

Use assistant moderators and make them co-hosts of the online call Assistants can help manage the technology while remaining muted with the camera off in the background. This can reduce cognitive burden for the moderator, allowing greater focus on the conversation. Ensure the assistant moderator role is explained to participants at the start of research events.

Designate personnel for emotional support In addition to an assistant moderator, a ‘runner’ or research assistant can act as a point of support for participants in difficulty. The role of this person should also be explained to all participants. Some participants may also wish to access support more discretely, and how this can be done should also be made clear.

Establish a culturally safe research space In any research, whether face-to-face or online, participants should feel culturally safe [24]. Managing the cultural safety of online interactions, particularly in group research, may sometimes be more challenging because visual cues that threaten cultural safety may be more difficult to read and respond to. Moderators need to establish ground rules early to set the tone and expectations of the room and be firm and decisive in using microphone control to temporarily mute disrespectful participants, or, in unresolvable situations, have an assistant remove them. Check that the selected technological platforms will allow the host to eject or temporarily mute a participant if necessary. Assistant moderators can also keep track of chat room interactions to help manage any challenging circumstances. While some online platforms (e.g. Zoom) can facilitate the provision of language interpretation via simultaneous audio channels, we note that ensuring cultural safety requires more than interpretation, and that adding additional channels does add technological and interpersonal complexity.

Manage microphones and background noise While asking participants to mute their microphones can often minimise background noise, having to turn the microphone on and off during interactions will also interrupt the flow of interactions. To maximise participation, leaving microphones on is recommended, despite the trade-off with background noise, which can interfere with data quality and the experience of other participants. Asking participants to do their best in minimising background noise or asking an assistant moderator to mute individual participants if background noise becomes problematic may help manage this. Discuss the preferred arrangement with participants at the start of the research event, including when and if microphones should be muted, and the best way to manage when to speak.

Have a back-up plan Sometimes technology can go wrong (computers crash, hardware malfunctions, internet connections go down), either halting the research or producing inaudible content. We have already considered the need for a clear, agreed backup plan to manage distress and cultural safety; this is also important to manage technical problems. Assistant moderators should hold a list of participants’ contact phone numbers and clear agreement with participants on when their contact number will be used. Moderators should be decisive about when to abandon the online platform and move to the back-up plan.

Manage unexpected intrusions ‘Zoom bombers’ join online meetings uninvited. They can cause interruption and embarrassment and they breach the privacy of a confidential research event. ‘Zoom bombing’ happens mostly when a link to the meeting is posted publicly and becomes searchable online. Use a private password for every online research event and consider using a waiting room for more control. Explicitly ask participants not to post events publicly or share links, and ensure passwords are secure and not publicised (e.g. on social media).

Conduct evaluation, and research online qualitative research Consider including questions about the use of the technology and online platform in post-research evaluations; feedback can not only be used to refine design and processes in future research but can also support methodological research.

Conclusions

Online methods were once marginal in qualitative research, rarely considered a first choice for data collection, and restricted mostly to those researchers who were interested in online worlds such as social media or gaming cultures as a subject of study. This has radically shifted. At the time of writing, the COVID-19 pandemic has driven much of everyday life into virtual worlds, as families, workplaces and existing social networks try to sustain themselves in the face of the risk of transmission. Niels Bohr allegedly quipped that prediction is very difficult, especially about the future; allowing for this caveat, we cannot imagine a future where everyday life or research practices return exactly to a 2019 pre-pandemic status quo. Online qualitative research has opened up a world of options for accessing participants and creating new types of data, and this seems likely to continue to expand. Qualitative researchers, then, need to respond to these new circumstances and opportunities in methodologically and ethically sound ways.

This paper is limited by our knowledge, experience and reading. Others will have expertise that we do not (e.g. in assistive communication technologies). We are also writing in a particular moment—a pandemic-induced flight to online research. As online qualitative research becomes mainstream, it is likely that technologies, practices and understandings will mature. Because change is inevitable, we have focused on principles rather than fine details of different platforms. There may be scope for researchers to engage with platforms over time and demand technological innovations that will more easily serve the ethical and methodological needs of research practice. Researchers themselves will also generate new qualitative methods that respond to the particularities of online platforms and their affordances. If researchers remain focused on design principles and attend to ethical, technical and social challenges, online methods will continue to make a significant contribution to qualitative health preferences research.

Author contributions

All authors, conceptualised, wrote, read, and approved the final manuscript.