Promises and pitfalls of data sharing in qualitative research

Abstract

The movement for research transparency has gained irresistible momentum over the past decade. Although qualitative research is rarely published in the high-impact journals that have adopted, or are most likely to adopt, data sharing policies, qualitative researchers who publish work in these and similar venues will likely encounter questions about data sharing within the next few years. The fundamental ways in which qualitative and quantitative data differ should be considered when assessing the extent to which qualitative and mixed methods researchers should be expected to adhere to data sharing policies developed with quantitative studies in mind. We outline several of the most critical concerns below, while also suggesting possible modifications that may help to reduce the probability of unintended adverse consequences and to ensure that the sharing of qualitative data is consistent with ethical standards in research.

1. Introduction

In 2014, the Public Library of Science (PLOS) journals unveiled a policy stipulating that authors must make available all data underlying the findings described in their published manuscript (Bloom et al., 2014). The implementation of this new policy was something of a watershed moment; although PLOS Medicine was not the first high-impact medical journal to require data sharing as a matter of policy, it is the only one that routinely publishes findings from qualitative studies and qualitative meta-syntheses. While the new guidance permits authors some latitude in circumventing data sharing, in some ways it does resemble the obligatory and much more rigorous conditions of publication already in place at leading journals in biostatistics (Peng, 2009), economics (Ashenfelter et al., 1986; Bernanke, 2004), and political science (Meier, 1995). At the American Economic Review, for example, authors make publicly available the raw data and statistical programming code needed to reproduce all of the findings in the published manuscript, and these materials are uploaded to the journal web site prior to publication (Bernanke, 2004). The experiences in these fields suggest that leading journals can implement unilateral changes that eventually contribute to building a culture in which data sharing becomes the norm.

The movement to promote reproducible research in the medical and public health literature has lagged, perhaps for myriad reasons. First, concerns are frequently voiced about intellectual property protections and/or the potential hazard of disclosing protected health information (Hrynaszkiewicz et al., 2010; Mello et al., 2013; Tudur Smith et al., 2015). Second, because medical and public health research can often carry enormous financial implications for specific products (Rennie, 1997; Shuchman, 2005) or entire industries (Kaiser, 1997; Michaels and Monforton, 2005; Muggli et al., 2001) that are implicated in the findings, requests for data may be driven by financial motivations that extend well beyond any disinterested concerns about science for science’s sake. A researcher might be appropriately wary, for example, of responding to an industry representative’s seemingly benign request for data. Finally, there are also structural barriers to data sharing, because faculty members at schools of medicine and public health are incentivized to publish secondary findings from a given data collection effort. For example, it is not uncommon for investigators to publish secondary analyses of data from randomized trials (Rotheram-Borus et al., 2015; Tsai et al., 2016) or multiple analyses of data from the same cohort (Colditz and Hankinson, 2005; Colditz et al., 1997). These concerns apply less strongly in the social sciences. Yet because this type of research often has direct relevance for patient care, data sharing should (in general) be regarded as an imperative for ensuring transparent analysis of data and reproducibility of research findings (Doshi et al., 2012; Le Noury et al., 2015).

The movement for research transparency has gained irresistible momentum over the past decade (Groves, 2010; Hanson et al., 2011; Laine et al., 2007; Miguel et al., 2014; Nosek et al., 2015; Peng et al., 2006; PLOS Medicine Editors, 2014; Stodden et al., 2013; Tsai, 2011). Although qualitative research is rarely published in the more high-impact journals (Greenhalgh et al., 2016; Shuval et al., 2011) that have adopted, or are most likely to adopt, data sharing policies, qualitative and mixed methods researchers who publish work in these and similar venues will likely encounter questions about data sharing within the years ahead, especially as mixed methods studies integrating qualitative and quantitative data become increasingly prominent (Creswell et al., 2011). The substantive ways in which qualitative and quantitative data differ should be considered when assessing the extent to which qualitative and mixed methods researchers should be expected to adhere to data sharing policies developed with purely quantitative studies in mind. We outline several of the most critical concerns below, while also suggesting possible modifications that may help to reduce the probability of unintended adverse consequences and to ensure that the sharing of qualitative data is consistent with ethical standards in research.

2. Reliability, validity, and reproducibility in qualitative research

2.1. Unique features of qualitative data production and analysis

Qualitative studies are based on data that are fundamentally different from the data collected in other observational study designs. The standardized measures employed in quantitative studies constrict the diverse perspectives of study participants along predetermined continua (e.g. categorical or continuous) so that they can be statistically aggregated. Quantitative data analysis plans (Olken, 2015) and study protocols (Horton, 1997) can be pre-specified and disseminated. The data can be anonymized and uploaded to secure data repositories. The statistical code used to process the data, and the process through which the output is translated into the manuscript text and tables, can just as easily be shared and replicated (Gandrud, 2013; Peng, 2009; Stodden et al., 2014; Vickers, 2006). External investigators can then use the electronic paper trail to verify the published findings (Dewald et al., 1986; Jefferson and Doshi, 2014; Le Noury et al., 2015; McCullough and Vinod, 2003). Data sharing, in effect, is “a threat that might keep potential cheaters honest” (p.722) (Hamermesh, 2007).

In contrast, the data collected in qualitative studies are typically obtained through in-depth interviews, focus groups, direct observation, document review, and audio recording review. These data, while typically not aimed at establishing generalizability, lend themselves to generating new theoretical insights about certain phenomena in greater depth and detail than is possible through quantitative designs (Patton, 2002). While complementary to other forms of social measurement, these data are also neither collected nor analyzed in as linear a manner, and it has been argued that the concept of reliability does not directly translate from the quantitative (rationalistic) to the qualitative (naturalistic) paradigm (Guba and Lincoln, 1981). In her influential essay, Stenbacka (2001) goes so far as to argue, “It is obvious that reliability has no relevance in qualitative research … If a qualitative study is discussed with reliability as a criterion, the consequence is rather that the study is no good” (p.552). The extremity of her viewpoint notwithstanding, more recent work in the field has sought to address questions about the validity and reliability of qualitative research findings, through the use of descriptive approaches (e.g., verification strategies (Morse et al., 2002)), quantitative approaches (e.g., calculating inter-rater reliability for comparing the assessments of multiple coders (Cohen, 1960) or proportional reduction in loss (Rust and Cooil, 1994)), and reporting checklists (Clark, 2003; O’Brien et al., 2014; Tong et al., 2007).

2.2. Reproducible research and qualitative data

For most readers of high-impact medical and public health journals, the term “reproducibility” will evoke the idea that external investigators ought to be able to arrive at the same published findings when given the data and analysis code (Claerbout and Karrenbach, 1992; King, 1995). In Clemens’ (in press) recently published typology of replication and robustness, this particular type of check is described as but one form of “replication” and given the label “verification”: “ensuring that the exact statistical analysis reported in the original paper gives materially the same results reported in the paper, either using the original dataset or remeasuring with identical methods the same traits of the same sample of subjects.” This definition corresponds closely to the concept of “methods reproducibility” suggested by Goodman et al. (2016). Notably, other researchers have ignored the distinction between “replication” and “reproducibility.” For example, the Open Science Collaboration (2012) have written: “Some distinguish between ‘reproducibility’ and ‘replicability’ by treating the former as a narrower case of the latter (e.g., computational sciences) or vice versa (e.g., biological sciences). We ignore the distinction” (p.659).

Verification does not translate well to a data sharing policy for qualitative studies. Given the inherently intersubjective nature of qualitative data collection, the iterative nature of qualitative data analysis, and the unique importance of interpretation as part of the core contribution of qualitative work, verification is likely to be impossible in the setting of qualitative research. We discuss two principal reasons below.

First, some scholars have argued that interview transcripts, even when accompanied by detailed field notes, cannot represent with sufficient fidelity the actual interview that took place. Even audio and video recordings, which are generally considered the most complete observational data that can be captured, cannot convey valuable tactile and/or olfactory data obtained in the field (Bernard and Ryan, 2009). Drawing on focus groups conducted with qualitative researchers, Broom et al. (2009) showed that many of them were of the immoderate opinion that their transcript data were “an encoded account only decipherable to the individual who collected it” (p.1170). According to this understanding, we should question the extent to which interview transcripts may be considered “raw data” for external investigators to use in the same manner as a dataset taken from a randomized controlled trial of the latest unoriginal antidepressant medication.

Second, the interview transcripts disseminated to external investigators are unlikely to be the data they would have collected had they conducted the study themselves. A qualitative study guided by the method of grounded theory, for example, follows an inductive process with concurrent review of the data being collected, filtering of the data for relevance and meaningfulness, and grouping and naming of patterns observed in the data (Glaser and Strauss, 1967). Investigators may also choose to collect additional data, if necessary, to deepen understanding into emerging phenomena via “theoretical sampling” (Glaser, 1978). Even if the authors of a particular study uploaded the entire set of field notes or interview transcripts to a secure data repository, what do these data mean to an external investigator who might not have the same kids of embedded cultural experiences (that would help contextualize the interview and field observation data) and who would have collected the data differently? An external investigator conducting a secondary analysis of a grounded theory dataset must be aware that, even if the same research questions are considered at the outset, s/he likely would have made very different decisions during the course of the study that would have led to an entirely different dataset being constructed. If external investigators perceive there to be gaps in the dataset they are provided by the study authors, the potential explanations for the missing data are legion: are data missing because the concepts of interest occurred too infrequently to be meaningful to the initial guiding propositions, because the phenomena were simply not present in the sample, or because the study authors’ interview probes were driven by a different conceptual lens?

Some researchers might view these unique features of qualitative modes of inquiry as befitting their position in the conventional “hierarchy” of evidence (Atkins et al., 2004; Guyatt et al., 1995). The economist Amitabh Chandra has quipped, for example, “If ethnography is a legitimate way to learn things … why aren’t [pharmaceutical] manufacturers allowed to do it?” (Chandra, 2015) Yet even quantitative data are subject to what Goodman et al. (2016) have labeled as “inferential reproducibility”: “… scientists might draw the same conclusions from different sets of studies and data or could draw different conclusions from the same original data, sometimes even if they agree on the analytical results” (p.4). Furthermore, it is important to note that secondary analyses of qualitative data would likely be able to reproduce at least some, if not all, of the major themes identified in the primary published article. However, that is not the aim of a verification test -- which is, rather, to reproduce “materially the same results reported in the paper” (Clemens, in press). Given these difficulties, it is likely that external qualitative investigators would not seek “verification” but rather “reproduction,” defined by Clemens as being another form of replication similar to verification except that reproduction studies are conducted with a different sample of study participants from the same population. This definition corresponds to the concept of “results reproducibility” suggested by Goodman et al. (2016). For example, Lewis’ (1951) re-study of the Mexican village Tezpotlán 20 years after Redfield (1930) might be considered, had it been conducted somewhat earlier, a reproduction test of a qualitative study. In theory, reproduction of a qualitative study does not require a data sharing policy. The authors’ description of the study’s methods, especially if guided by a reporting checklist (Clark, 2003; O’Brien et al., 2014; Tong et al., 2007), should be sufficient to enable another team of investigators to conduct a reproduction test. But if reproduction, rather than verification, is the goal, then of what relevance is a data sharing policy?

3. Data sharing in qualitative research

Beyond attempts to increase transparency in the production of qualitative data, it is likely that qualitative and mixed methods researchers will need to address qualitative data sharing in some fashion. Applying these standards uncritically, one might presume that data sharing involves providing the following in an online supplementary appendix: interview guides and interview transcripts, in the original language and in the translated language of the investigators (if different from the original); field notes; data used, if any, to establish inter-coder reliability; full code books; and documents, if any, describing the process of open coding, selection of codes for inclusion in the final codebook, and category construction. The “audit trail” supports reliability and validity, so even if it is recognized that no two groups would conduct identical qualitative studies, the information available to external investigators would enable them to understand how the study authors arrived at the published conclusions. Most computer-assisted qualitative data analysis software packages offer export functions that enable users to save an entire “project” (e.g., raw data, codebook, coding links, and memos), which could facilitate dissemination. While these types of maneuvers might be consistent with a data sharing policy, there are a number of challenges that could hamper their implementation in practice. Below we highlight the most significant challenges facing data sharing in qualitative research.

3.1. Preserving the anonymity or pseudonymity of study participants

Data sharing policies should carefully consider the potential effects of data sharing on study participants. Most qualitative researchers use respondent validation (e.g., reviewing emerging themes and analyses with study participants or key informants) to ensure rigor, and the practice is highlighted as a key process component of qualitative research in most reporting checklists (Clark, 2003; O’Brien et al., 2014; Tong et al., 2007). This method of data sharing through member-checking of interim findings is carefully supervised. In contrast, data sharing policies that make interview transcripts available to study participants in a completely unstructured fashion may have negative effects. Chief among these are the potential psychosocial consequences of compromising study participant anonymity.

Because qualitative study designs often lend themselves to the in-depth study of highly sensitive subject material (Kelly et al., 2011; King et al., 2013; Parkinson, 2013; Wade et al., 2005), field notes and interview transcripts would need to be anonymized prior to dissemination in order to conform with prevailing legal and ethical guidelines. Institutional Review Board concerns about participant anonymity, discussed in the PLOS policy (Bloom et al., 2014), have been identified as a leading barrier to data sharing. Consequently, investigators lacking proper guidance on how to comply with data sharing guidelines in a way that provides adequate anonymity protections may simply default to data withholding. For example, in the Data Availability Statement for their qualitative study recently published in PLOS Medicine, Christopoulos et al. (2015) stated, “Public availability of data could potentially compromise participant privacy. Participants did not consent to have their full transcripts or excerpts of transcripts made publically [sic] available.” Qualitative studies published in PLOS One subsequent to the PLOS policy adoption have made similar claims (Natoli et al., 2015; Tang et al., 2015) (although there have also been notable, and welcome, exceptions (Lo et al., 2016)).

While Institutional Review Board restrictions are commonly cited to justify withholding of quantitative data (Campbell et al., 2002), in fact it may be possible to release de-identified versions of transcripts that preserve the anonymity of qualitative study participants. The nature of any anonymization procedures would depend on the nature of the data collected and the extent to which the data can be linked with publicly available information to reveal specific identities. At a minimum, the anonymization procedures would entail redaction or alteration of protected health information and any specific encounter details that reveal, however indirectly, the identity of any of the parties to the encounter, with obfuscated information shown in brackets. The investigator might keep a detailed record of these procedures in a secure location should it become necessary to revisit the data after publication (Table 1), similar to the recommendations made in the Privacy Certificate Guidance of the U.S. National Institute of Justice (2007). As a cautionary note, depending on the size of the dataset, the redaction or anonymization process could require tremendous time and effort of the investigators and could also potentially introduce errors and inconsistencies (Goffman, 2014; Lewis-Kraus, 2016). Additionally, for some studies, the nature of the research (Parkinson, 2013) may be such that any suitably redacted or anonymized transcripts might be so unserviceably thin that they would be devoid of meaningful content. Wolcott (1973) discusses this possibility in the introduction of his classic ethnography: “To present the material in such a way that even the people central to the study are ‘fooled’ by it is to risk removing those very aspects that make it vital, unique, believable, and at times painfully personal” (p. 4).

Table 1.

Supplementary information table of procedures taken to anonymize or redact hypothetical interview transcripts prior to dissemination in a data repository (N = 53).

Study participant	Line number	Original	Anonymized
Clinic patient 2	79	“My husband has been beating me regularly since I was married to him at age 18”	“My husband has been beating me regularly since I was married to him at [a young age]”
Clinic patient 2	85	“Just the other day he got angry with me because there was no water and our eldest went to school in a soiled uniform. He threw the empty jerricans at me and you now see the bruise on my left eye”	“[ ] He got angry with me because there was no water [ ]. He [attacked me] and you now see [my face]”
Community member 8	243	“I am the headmaster of the Buhingo Boarding School. What would the parents say if they knew I was HIV positive?”	“I am the headmaster of [a school]. What would the parents say if they knew I was HIV positive?”
Clinic patient 53	164	“I was in the hospital for a week after injuring my left leg in a boda boda accident. The nurse at the Mbarara Hospital chastised me when she found out my HIV status.”	“I was in the hospital [after a transportation accident]. The nurse [ ] chastised me when she found out my HIV status.”

Because interview transcripts contain verbatim quotations, it is likely that some transcripts cannot be sufficiently anonymized to prevent deductive disclosure, or what Tolich (2004) has called violations of “internal confidentiality.” That is, study participants could recognize themselves, their communities, or other study participants (if they belong to the same community) (Larossa et al., 1981). van den Hoonaard (2003) holds that anonymity is “a virtual impossibility in ethnographic research” (p.141). Depending on the sensitivity of the subject matter, deductive disclosure could result in harm to study participants and their relationships with others in the community. Ellis, 1995; Scheper-Hughes, 2000, and Stein (2010) have famously written about being angrily received by study participants over deductive disclosures following the publication of their celebrated books (Ellis, 1986; Scheper-Hughes, 1977; Stein, 2001). If such aggravated harm could result from the publication of books and journal articles in which verbatim quotations are carefully curated, one can imagine the harm resulting from a data sharing policy requiring entire interview transcripts to be shared.

Certain types of studies may carry even greater risks of deductive disclosure. These include studies of small-scale societies; studies that rely on respondent-driven sampling and other variations of snowball sampling to identify hard-to-reach populations; and studies in which permission to access a small community must be first secured from highly networked research gatekeepers, such as village leaders or community advisory boards. In these settings, a minor, idiosyncratic detail -- such as a manner of speaking or a specific phrase -- that is of unknown significance to the investigator (and therefore likely to go unredacted) could result in deductive disclosure and potential harm. In addition to the risk of harm to study participants, deductive disclosure also raises important questions about potential risks to third-party non-participants when study participants disclose sensitive information about social network ties that arises from their shared history with others (Larossa et al., 1981; Lounsbury et al., 2007; McLellan et al., 2003).

Related to the above, data sharing potentially further limits qualitative research done through “studying-up” (Nader, 1969) or “studying over” (Markowitz, 2001) -- approaches in which persons in positions of power (e.g., hospital administrators, pharmaceutical company executives, heads of governmental or multilateral organizations) become the subject of ethnographic study (Abramowitz and Panter-Brick, 2015; Closser, 2010). Because elites are more empowered to articulate concerns about confidentiality and disclosure, data sharing could unintentionally perpetuate power differentials in which health program beneficiaries endure as research subjects while health program funders and implementers remain understudied (Schneider and Aguiar, 2012).

Given the greater risks of deductive disclosure through unregulated data sharing (as contrasted with the carefully curated release of specific quotations through publication of study findings), consent documents for qualitative studies would need to properly inform prospective study participants that the interview transcripts could potentially be uploaded to a shared data repository for public consumption. Even researchers who have no intentions to share the data might be advised to seek informed consent from study participants at the outset simply to preserve the option in the future (Groves, 2010). Although study participants’ exposure to such risk would ultimately be contingent on the researchers’ decision to publish their findings in a journal where a data sharing policy is enforced, it is likely that such a caveat -- however conditional -- would result in selection on unobserved heterogeneity. These selective pressures could shape the types of persons who agree to participate in qualitative and mixed methods studies; alternatively, these selective pressures could have no impact on the types of persons who agree to participate but could shape the nature of the data they are willing to share with investigators. Either of these selective pressures would likely compromise the quality of the research, thereby upending one of the distinctive advantages of qualitative research, which is the ability to conduct in-depth examinations of sensitive subject material (Kelly et al., 2011; King et al., 2013; Parkinson, 2013; Wade et al., 2005).

To minimize the risk of deductive disclosure, a data sharing policy might, in lieu of obliging the release of interview transcripts, require investigators to implement procedures to enhance transparency. Many aspects of the qualitative analysis (e.g., transcription rules, data segmentation, coding units, process for code development, finalized codes) could be shared with minimal risk to study participants. Taking transparency a step further, investigators could export coding queries and make these available to external investigators. Because coding queries consist of excerpted and possibly disembodied interview text, they may offer greater anonymity compared with full transcripts. Depending on the interview content, investigators may still need to redact some of the text to preserve anonymity -- which would entail added burden -- but the risk of deductive disclosures would be reduced. The release of coding queries has not been suggested in the ongoing conversation on data sharing in qualitative research but should be regarded as a viable and potentially more ethical way to promote transparency than the release of full transcripts.

An example of a coding query, applied to data from Kohrt et al. (2010) and Morley and Kohrt (2013), is provided in the Electronic Supplementary Appendix. Coding queries would provide external investigators with comprehensive information that could be used to qualitatively assess the internal coherence of the coding scheme (Box 1). In qualitative research, study participants often present conflicting or contradictory views on the same topic based on varying influences such as the nature of the question and the time elapsed during the interview (LeCompte and Schensul, 1999). Discrepant data may be especially important in longitudinal studies where study participants provide serial interviews during the course of an illness or throughout their lifetimes, thereby gaining increasing familiarity with a particular interviewer. These processes are rarely, but with some exceptions (Groleau et al., 2006), captured in academic publications that tend to present views as static and internally coherent. Ultimately, much like the sharing of data from quantitative studies can provide opportunities to conduct detailed interrogations of the scientific record (Le Noury et al., 2015), coding queries can help reviewers and external investigators assess whether the quotes provided in manuscripts and journal articles capture the overall content of the data or whether they represent selective reporting of study participants’ perspectives in a way that suits the authors’ theses.

Box 1. Using coding queries to evaluate the internal coherence of the coding scheme.

1. Do the quotes represent similar concepts to a sufficient degree to justify a coherent theme?

2. Is the concept shared among study participants throughout the sample, or is it limited to specific subset? If limited to a specific subset, is the circumscribed nature of the concept adequately described in the manuscript?

3. Does the description or valence of the concept change during the course of the interview or during the course of multiple interviews with the same study participant? If so, are these changes adequately described in the manuscript?

4. Does the choice of quotes, and their accompanying descriptions, presented in the manuscript adequately capture the content and diversity of the coding query?

3.2. Other unintended consequences of qualitative data sharing

In addition to the risk of deductive disclosures, a number of other unintended consequences could result from data sharing policies if they are not properly tailored to the unique aspects of qualitative and mixed methods research. First, the burden of organizing qualitative data for inspection or use by external investigators could easily exceed the work of writing the manuscript itself. How should the interests of research transparency be weighed against the potential costs of documentation burden? Redacting the hundreds of pages of transcripts collected during the course of a small qualitative study would require months of work. Moreover, there are no standards in the field for systematically documenting the hours of conversations, conference calls, and e-mail exchanges required for code selection and category construction. Guidelines would need to be developed so that documentation of these procedures is uniform across studies. Larger qualitative and mixed methods studies would entail an even greater documentation burden. For example, the longitudinal qualitative study by Maman et al. (2014) involved 657 study participants and 1059 in-depth interviews, with each interview averaging 30–60 minutes in duration. Even redacting just the 175-page summary reports for each of the 48 sites -- much less the primary interview transcripts -- would have required the review of more than 8000 pages of data. In what format should such data be made available to meet the conditions of a reasonable data sharing policy?

Second, and related to the above, journals should consider the possibility that, in response to data sharing policies, study participants and qualitative researchers may alter their behavior in undesirable ways. Will qualitative researchers, whose work is already de facto excluded from most high-impact journals (Greenhalgh et al., 2016; Shuval et al., 2011), shy away from submitting their work to these journals, where data sharing policies are increasingly enforced? Will they be discouraged from conducting large-sample qualitative studies, knowing the documentation burden that will be involved? Furthermore, it is one thing to make available several hundred pages of interview transcripts from a two-to three-year qualitative study conducted by paid research assistants. It is another thing to make available thousands of pages of field notes and journal entries -- some of which may be intensely personal in content -- accumulated during the course of a five-year ethnography. Ethnographic note-taking guidelines that separate field notes according to observation, interpretation, and personal reflection (Bernard, 2006) could potentially facilitate data sharing by restricting dissemination to material related to observation. Unless qualitative researchers have a secure understanding that certain types of material can be shielded from dissemination, they may be motivated to alter the underlying data, i.e., by withholding this material from the written or transcribed record (Baez, 2002; Goodwin et al., 2003; McLellan et al., 2003; Scheper-Hughes, 2000) or by maintaining a set of private “shadow files” separate from the official research record (similar to the detailed “psychotherapy notes” that therapists store apart from the medical record).

Box 2 summarizes our recommendations for journal policies that would promote transparency and, in some cases facilitate sharing of qualitative data, while remaining sensitive to their unique attributes that require their distribution to be handled somewhat differently than quantitative data.

Box 2. Summary of recommendations for journal editors.

1. Require a statement from authors about whether the consent process included a description of any public availability of data. Prior to public dissemination of data, authors should provide a statement to journal editors about whether or not study participants were informed about future plans for public availability of data and the manner in which this issue was addressed, if at all, during the informed consent process.

2. Require adherence to minimum standards for de-identification of publicly shared data. Under the 1996 U.S. Health Insurance Portability and Accountability Act, protected health information includes 18 identifiers (e.g., names, addresses, serial numbers) that must be treated with special care in quantitative datasets. These same identifiers should be removed from qualitative data prior to dissemination. The geographic subdivision requirement, which stipulates that geographic units contain 20,000 or fewer people, requires special attention. If qualitative researchers are working in a village or community with fewer than 20,000 people, then site pseudonyms or larger geographic divisions should be used in published reports (e.g., providing the sub-county name rather than the parish or village name).

3. Encourage authors to use, and publish, data from multiple informants and/or institutions per selection category. Whenever possible, authors should be encouraged to recruit more than one informant and more than one institution per category. For example, interviewing only one surgeon at a hospital or only one official at a ministry of health increases the probability that the study participant’s comments may be traced back to that study participant (or study participant’s institution). If two or more informants are recruited per selection category and a range of institutions are included, the probability of identification may be reduced. Journal policies related to this provision should be cognizant of the lesser amounts of funding granted for qualitative research and the smaller scale of qualitative studies.

4. Permit coding queries to be shared as an alternative to full transcripts. Coding queries may offer greater anonymity compared with full transcripts because statements are grouped by theme rather than by study participant. Furthermore, coding queries allow a form of verification of the findings reported in results and conclusion. For the purposes of promoting transparency in qualitative research, these should be considered acceptable, or possibly even preferable, alternatives to full transcripts.

5. Encourage anonymization of field notes. Ethnographers frequently rely on field notes as a source of data. These could be anonymized in the same fashion as interview transcripts before being made publicly available. Because field notes include a range of objective, subjective, and interpretative documentation, requests for field notes should be limited to objective excerpts. Field notes, as with other forms of qualitative data, could also be submitted in the form of coding queries, with the same advantages as discussed above.

6. Encourage authors to document social audits or other stakeholder dissemination at the time of manuscript submission. A major source of participant-researcher dispute occurs when participants feel that their responses are selectively represented in the reported results or in recommendations drawn from the data. Public availability of qualitative data may therefore be especially contentious if study participants, or their representatives (e.g., local leaders), have not signed off on the researchers’ interpretations. Social audits or other stakeholder dissemination of results and conclusions prior to public availability of data will foster participants’ perceptions of inclusiveness and accurate representation.

7. Encourage manuscript reviewers with requisite expertise in qualitative and mixed methods research to comment on the adequacy of anonymization. Study authors are ultimately responsible for anonymization. However, to promote good scientific practice, journal editors should encourage manuscript reviewers with requisite expertise in qualitative and mixed methods research to comment on the adequacy of anonymization and to raise any concerns they may have regarding potential maleficence resulting from data sharing.

8. Establish a petitioning process for non-disclosure of data. Authors should have the option of petitioning for non-disclosure of qualitative data in select instances. These include scenarios in which the study could not have yielded important results if participants were to have been required to consent to public disclosure of data, or in which anonymization could not be adequate given the uniqueness of the study population or the data.

4. Conclusion

Data sharing in medical and public health research is becoming increasingly normative, but medical and public health journals have yet to grapple with how to feasibly and ethically promote data sharing for qualitative and mixed methods research. Recent advances in the field have begun to enhance the reliability and validity of qualitative data. Data sharing may help to increase confidence in qualitative research findings, but the concept of reproducible research does not translate as straightforwardly from quantitative data to qualitative data. Data sharing policies may be feasible for qualitative studies, but leading medical and public health journals should consider modifying their policies to be more relevant to the unique aspects of qualitative and mixed methods study designs; they must also address concerns about potential violations of participant anonymity and other unintended adverse consequences. Such policies, if appropriately implemented, can build a culture of data sharing that also facilitates critical, patient-oriented qualitative and mixed methods research.

Appendix A. Supplementary data

The following is the supplementary data related to this article: