Are we ready to share qualitative research data? Knowledge and preparedness among qualitative researchers, IRB Members, and data repository curators

Abstract

Data sharing maximizes the value of data, which is time and resource intensive to collect. Major funding bodies in the United States (US), like the National Institutes of Health (NIH), require data sharing and researchers frequently share de-identified quantitative data. In contrast, qualitative data are rarely shared in the US but the increasing trend towards data sharing and open science suggest this may be required in future. Qualitative methods are often used to explore sensitive health topics raising unique ethical challenges regarding protecting confidentiality while maintaining enough contextual detail for secondary analyses. Here, we report findings from semi-structured in-depth interviews with 30 data repository curators, 30 qualitative researchers, and 30 IRB staff members to explore their experience and knowledge of QDS. Our findings indicate that all stakeholder groups lack preparedness for QDS. Researchers are the least knowledgeable and are often unfamiliar with the concept of sharing qualitative data in a repository. Curators are highly supportive of QDS, but not all have experienced curating qualitative data sets and indicated they would like guidance and standards specific to QDS. IRB members lack familiarity with QDS although they support it as long as proper legal and regulatory procedures are followed. IRB members and data curators are not prepared to advise researchers on legal and regulatory matters, potentially leaving researchers who have the least knowledge with no guidance. Ethical and productive QDS will require overcoming barriers, creating standards, and changing long held practices among all stakeholder groups.

Background

Data is a valuable commodity that is costly and labor-intensive to collect (Mannheimer et al., 2018). Data sharing is one way to maximize the value of data. Benefits of data sharing include avoiding duplication of research, enabling secondary analyses, reducing research participant burden, and providing training resources to students (Mannheimer et al., 2018, Corti, 2000, Corti, 2012, DuBois et al., 2018, Borgman, 2012). Data sharing also promotes transparency and enables replication of findings. Major United States (US) funders such as the National Institutes of Health (NIH) and National Science Foundation (NSF), select journals, and private funders like the Gates Foundation, all have policies requiring some form of data sharing (Mannheimer et al., 2018, Meyer, 2018). The NIH ‘expects and supports the timely release and sharing of final research data from NIH-supported studies for use by other researchers’ (National Institutes of Health, 2003). While the NIH recognizes that data sharing may be complicated or limited by ‘institutional policies, local IRB rules, and local, state and federal laws and regulations, including the HIPAA Privacy Rule’ they endorse data sharing wherever possible. In some cases, NIH mandates data sharing or an explanation to justify why data cannot be shared (Meyer, 2018,National Institutes of Health, 2003). Even when not required, there is arguably an ethical obligation to share data collected using public funds (DuBois et al., 2018, Meyer, 2018).

Notably, these policies do not specify the type of data to be shared, although most were likely written with quantitative data sharing in mind. There is, however, no reason to assume these policies may not apply equally to qualitative data. In fact, the increasing trends towards data sharing and open science suggest that qualitative researchers may find themselves subject to data sharing requirements (Meyer, 2018, Yoon, 2014, Tsai et al., 2016, Mannheimer et al., 2018, DuBois et al., 2018).

Differences between qualitative and quantitative data raise challenges–both epistemological and ethical–to sharing qualitative data (Tsai et al., 2016, Guishard, 2017b, Mannheimer et al., 2018, Yoon, 2014). Ethically, concerns exist regarding informed consent, data ownership, confidentiality, and adequately anonymizing qualitative data while maintaining enough contextual details to enable secondary analyses. Qualitative methods are frequently used to explore highly sensitive or stigmatized issues, which increases the harmful effects of potential re-identification of participants (Tsai et al., 2016, Guishard, 2017a).

The Health Insurance Portability and Accountability Acts (HIPAA) Privacy Rule regulates the collection, handling, sharing, and transfer of Protected Health Information (PHI). HIPAA provides two methods to de-identify PHI: removal of 18 ‘safe Harbor’ identifiers³ or expert determination, although other anonymizing tools exist (HIPAA 2017, Meyer, 2018). In practice, removal of the 18 Safe Harbor identifiers is commonly used as it is easy to operationalize. However, qualitative data present unique challenges for anonymization that will require more sophisticated tools than ‘rote application of HIPAA Safe Harbor rules’ to ensure adequate anonymization (Meyer, 2018). For instance, three demographic variables may not appear to be identifiers when examined independently: ‘female’, ‘Hispanic’, and ‘psychiatric nurse’ (and none are considered HIPAA Safe Harbor identifiers). However, if a researcher publishes the hospital where the research occurred, then it may become relatively easy for individuals who work at the hospital to identify this person (DuBois et al., 2018). To complicate matters further, qualitative health data may not be considered PHI when gathered in a research setting, and therefore may not be subject to HIPAA, leaving it in a ‘regulatory twilight zone’ (Meyer, 2018).

Epistemologically, researchers may not view qualitative data that are removed from the original context as being appropriate for analysis by third parties who lack the original researcher’s contextual knowledge and expertise (DuBois et al., 2018, Mannheimer et al., 2018, Tsai et al., 2016). Qualitative researchers have expressed concerns that their data are often collected within relationships of trust and that secondary analyses by new researchers are neither feasible nor ethically acceptable (Guishard, 2017a). For some, the complexity of qualitative data simply does not lend itself to reuse or sharing (Yoon, 2014). The notion of replicating qualitative data may also be problematic because qualitative studies are rarely meant to be representative or generalizable (DuBois et al., 2018). It is not suitable to speak of replicating qualitative data in the same manner as replicating quantitative data. However, sharing codebooks, interview guides, and transcripts would allow others to verify the rationale for the claims made in a particular qualitative study (DuBois et al., 2018). Further, secondary analyses of qualitative data can yield new insights (Bishop and Kuula-Luumi, 2017).

These challenges to qualitative data sharing (QDS) may be especially pertinent in the US where QDS is relatively new (Yoon, 2014, Corti, 2000, Corti, 2012). In Europe and Australia, policies to promote and encourage QDS have existed for some time, and researchers are more familiar with the concept (Corti and Backhouse, 2000, Kuula, 2011, Yoon, 2014). Repositories such as the UK Data Archive or Finnish Social Science Data Archive provide national storehouses for social science and humanities data (Corti and Backhouse, 2000, Kuula, 2011, Yoon, 2014). Data repositories exist in the US, but most are not capable of handling sensitive qualitative data (Antes et al., 2017, Mannheimer et al., 2018). Some repositories may only be able to take data that will be made available open access, prohibiting them from accepting sensitive data (Antes et al., 2017). A review of qualitative data repository guidelines found 32 English language social science repositories globally; only 12 repositories had written guidelines for qualitative data, and only 5 were located in the US (Antes et al., 2017).

The limited available data suggests US social scientists rarely share their data (Jeng et al., 2016). A survey of over 1,200 researchers regarding data sharing practices found that those in social sciences and medicine were the least willing to share data compared to disciplines such as atmospheric science and biology (Tenopir et al., 2011). This is related to the fact that these data are more likely to be sensitive human subject data. Those in the social sciences also reported the lowest level of institutional support for data management, including the necessary processes, tools, funding, and technology (Tenopir et al., 2011).

The Need for Our Project

In light of the relative newness and potential expectation of QDS in the US, research is needed to explore diverse stakeholder experiences of and attitudes towards QDS. We received a grant from the NIH to explore the barriers and facilitators to QDS. We conducted 120 semi-structured in-depth telephone interviews with 4 stakeholder groups: 30 qualitative research participants, 30 data repository curators, 30 qualitative researchers, and 30 IRB staff members to explore knowledge, barriers, and benefits of QDS. While qualitative research can take many forms, we focus on qualitative health data; that is, textual data gathered in an interview or focus group that may include potentially sensitive health information. Focusing on transcribed data provides a useful starting point as they are common, useful, and may be easier to anonymize than audio or video files that contain voice prints or facial images (DuBois et al., 2018).

Methods

Data collection

We report findings from qualitative interviews and pre-interview surveys with three stakeholder groups—qualitative researchers, IRB staff, and data curators—regarding their preparedness, or lack thereof, for QDS. Findings from the research participant stakeholder group will be reported in a forthcoming article.

We built recruitment lists using publicly available databases or personal contacts supplemented by snowball recruitment. Data curators were identified through the Open Access Directory (Open Access Directory, 2016). Curators were eligible if they worked at a repository that could accept multi-disciplinary or social science data, although not all repositories had experience curating qualitative data sets. Due to the limited number of qualitative repositories globally, we included curators from international social science repositories in addition to US curators (Antes et al., 2017). IRB members and staff were identified through the websites of 62 academic medical institutions with NIH Clinical and Translation Science Awards (CTSA). IRB members and staff had to have experience reviewing and approving qualitative research studies. Qualitative researchers were identified through personal contacts and publicly available information from organizations that represent minority researchers. We purposively sampled qualitative researchers to ensure one-third of the sample was from racial or ethnic minorities. Researchers were eligible if they were the principal investigator or lead of a project involving qualitative data collection. All participants had to speak English.

The research was approved by the Washington University School of Medicine Institutional Review Board (IRB) as expedited human subjects research.

Recruitment

We recruited participants via email invitation. After providing informed consent, participants completed a brief survey online administered via Qualtrics to obtain demographic details and prior experience before completing a semi-structured telephone interview. Interviews explored prior experience and knowledge, perceived barriers and benefits, and preparedness for QDS. Questions ranged from broad and open-ended such as ‘Tell me what you know about QDS’ to more focused questions about particular issues we anticipated would arise such as ‘What concerns do you have about cost?’ Interview guides for each stakeholder group followed a similar structure, but questions were adapted to be relevant for each stakeholder group. For instance, we asked all three stakeholder groups about their preparedness for QDS; researchers were asked about preparedness to deposit qualitative data with a repository, while the question for IRB members and data curators was posed as preparedness to advise researchers regarding QDS. Participants were paid $30 USD following completion of the interview. Trained interviewers (Parsons, Baldwin, Walsh) conducted the interviews, which lasted approximately one hour.

Data Analysis

Interviews were audio recorded and professionally transcribed verbatim before being uploaded to Dedoose, a qualitative data analysis software.⁴ Survey data were exported from Qualtrics into Dedoose and linked to the relevant transcript. The Principal Investigator, Co-investigator, and project manager (DuBois, Mozersky, Walsh) led codebook development for each stakeholder group with input from all interviewers. Coding involved a combination of inductive concept coding and deductive structural coding (Saldana, 2016). Inductive concept coding was used to capture participant views and attitudes about QDS that arose spontaneously among stakeholder groups (usually in response to broad, open-ended questions although not exclusively). Structural deductive coding was used to categorize answers to the specific questions regarding the hypothesized benefits and challenges we anticipated based on the limited literature (e.g., concerns about confidentiality or cost). We assigned one coder to each stakeholder group, enabling them to become familiar with the particular data set and codebook. Coders kept detailed notes and memos during coding. One author (Mozersky) served as the gold standard coder for all stakeholder groups. In the first step, the coder and gold standard coder blind coded a single transcript, discussed discrepancies, and resolved them. We repeated this process until both coders reliably coded without major discrepancies. We held weekly meetings with coders to discuss questions or discrepancies. For each stakeholder group, the gold standard coder periodically blind coded interviews to ensure agreement between the gold standard and coders. This blind coding occurred for ~15% of each set of 30 interviews (i.e., 4 – 5 transcripts per set). Discrepancies arising during blind coding were discussed with coders and resolved through consensus.

Results

We present our results by stakeholder group, but all groups demonstrate a lack of knowledge and experience with QDS. Tables I, II, and III contain demographic details and prior experience for each group. Researchers are the least knowledgeable about QDS and were often unfamiliar with the very idea of sharing qualitative data with a repository. Curators are highly supportive of QDS, but not all had experienced curating qualitative data sets. Furthermore, many indicated they would like guidance and standards specific to sharing qualitative data. IRB members lacked familiarity with QDS plans, as they often had not encountered them. However, in principle, they did not see a problem with QDS as long as proper legal and regulatory procedures are followed and that sharing is consistent with the information provided in the informed consent document. These findings are not surprising—QDS is relatively new, uncharted, and many have not yet experienced it.

Lack of Knowledge

Among qualitative researchers, only one individual had experience depositing qualitative data with a repository (Table I). Four individuals were aware of a peer who had used a qualitative repository. In the early phase of coding, the team created a code for lack of knowledge to capture demonstrations that researchers did not have knowledge or understanding of QDS. We created this code because it was evident upon reading transcripts that lack of knowledge permeated researchers’ responses. Researchers’ responses were often speculative and did not reflect actual experience with QDS. This code captured a wide range of sentiments from having never heard of a repository to being familiar or curious about their existence but not knowing the details of how repositories operate.

Table I:

Researchers (n=30)

Demographic	Frequency	Percenta
Age
20–29	0	0%
30–39	6	20%
40–49	14	47%
50–59	6	20%
60 or older	4	13%
Sex
Female	26	87%
Male	4	13%
Raceb
Asian	1	3%
Black or African American	7	23%
White	20	67%
Prefer not to answer	1	3%
Ethnicity
Hispanic or Latino	2	7%
Not Hispanic or Latino	28	93%
Region of Birth
United States	28	93%
Africa	1	3%
Asia	1	3%
Education
Doctoral Degree	28	93%
Other Degree	2	7%
Degree Field
Anthropology	4	13%
Communications	2	7%
Psychology	6	20%
Public Health	3	10%
Social Work	3	10%
Other	8	27%
Academic Rank
Instructor	1	3%
Assistant Professor	10	33%
Associate Professor	8	27%
Full Professor	7	23%
Other	4	13%
Years’ Experience
0–2	1	3%
3–5	3	10%
5–10	9	30%
10 or more	17	57%
Collecting Sensitive Datab
Personal Health Information (PHI)	19	63%
Sensitive non-PHI data	17	57%
Populations Studiedb
Healthy Individuals	24	80%
Patients	15	50%
Children	7	23%
Pregnant Women	3	10%
Older Adults	9	30%
Individuals with sensitive diagnoses	10	33%
Economically disadvantaged individuals	21	70%
Prisoners	4	13%
Other	13	43%
Methods Usedb
In-Depth Interviews	30	100%
Focus Groups	25	83%
Observations	18	60%
Community Based Participatory Research (CBPR)	15	50%
Coding of Archival Data	9	30%
Journals written by participants	2	7%
Other	3	10%
Has Shared Data with a Repository
Yes	1	3%
No	29	97%
Knows a Peer who has Shared Data with a Repository
Yes	4	13%
No	26	87%

^aNumbers may not add up to 100% due to rounding

^bParticipants were asked to check all that apply so percentages will not equal 100%

When asked what they know about sharing qualitative data in a repository, some researchers were entirely unfamiliar with the concept, as this researcher indicated, ‘I’m not aware that there is such a thing…’ [R1]. Others associated data sharing with quantitative data:

I actually never thought about it before. When I hear ‘data repository,’ I think quantitative data, so I never even considered using it for qualitative

[R3].

For some respondents, taking part in the interview for our study was the first time they had encountered the idea of QDS. As this researcher notes:

We’ve never done it…I don’t know what the process is, or who gets to access it, or how it needs to be deidentified…other than what you described earlier

[R23].

Other researchers were familiar with the concept and open to the idea of sharing qualitative data but lacked practical knowledge in terms of how, where, or what is actually involved in the process. This researcher stated:

My first impulse was, ‘Yeah, I could do that,’ and then, you know, I don’t know which data repositories are good. I don’t know what the requirements are. Who knows what the IRBs going to want and how participants are going to react

[R8].

Some researchers were uncomfortable with the idea of sharing qualitative data at all, and lack of knowledge played a role in these concerns. As this researcher stated:

Personally…I don’t feel comfortable sharing my data in a data repository…and I’m not sure what would be in the repository, whether it would just be, you know, interview transcripts, or would it be field notes as well? What would be the kind of data that would be placed in a repository?

[R25].

Lack of familiarity also leads to a lack of trust. According to this researcher, ‘I’ve never worked with those people, so I don’t even know, can I trust them with the data? That’s a big question’ [R22].

Researchers frequently expressed a more fundamental concern regarding the purpose or value of QDS. That is, they first need to be convinced about the benefits before considering placing data in a repository. As this researcher asked ‘Why does this make any sense to do? Why? Why? ‘ [R29]. Another researcher suggested that qualitative data is shared through journal articles, literature, and intellectual discourse and that a repository was ‘a very narrow notion’ of sharing qualitative data. This attitude is noteworthy because data presented in the literature or at conferences is arguably highly curated and edited, leaving the majority of data sequestered from other researchers or the public (DuBois et al., 2018).

Preparedness for QDS

We asked researchers how prepared they felt to share qualitative data, and the majority reported they were not prepared for two primary reasons: either 1) they did not see the value or benefit in sharing qualitative data and were resistant, or 2) they did not have enough knowledge regarding how to deposit qualitative data. This researcher stated, ‘I feel prepared, but I feel skeptical’ [R21]. According to this researcher:

I would need an incentive to do it, like somebody would have to either pay me or, there would have to be, like, real strong norms in the field that this is what people do…

[R11].

Another researcher commented, ‘Publication is important for me, and I need all this data without sharing…it…’ [R9].

In contrast, other researchers were open to the idea, but they reported not knowing how to go about sharing their data. According to this researcher, ‘I’m interested in doing it…I haven’t had experience doing it’ [R6]. Another researcher stated, ‘I’ve thought through some of the issues, but I’ve never actually had to do it and think through the logistics.’ [R27] This researcher said, ‘I don’t even know…what my university’s stance is on it, so I would say no. I’m not very prepared at all’ [R13].

While researchers expressed a range of attitudes about QDS from supportive to curious to highly suspicious, all researchers lacked knowledge regarding QDS, repositories, and the potential to share qualitative data via a repository.

Lack of Knowledge

Among IRB members and staff, just over half (16/30) had reviewed a study with a QDS plan (Table II). We also created a lack of knowledge code to capture demonstrations of lacking knowledge regarding QDS, available repositories, permissions required, data security and access, and references to wanting guidelines, or standards. In contrast to researchers whose lack of knowledge contributed to their concerns about QDS, IRB members supported QDS as long as sharing was compliant with regulations and participants had provided informed consent. However, their responses demonstrated a lack of familiarity with the details of QDS in practice.

Table II:

IRB (n=30)

Demographic	Frequency	Percenta
Age
20–29	4	13%
30–39	9	30%
40–49	5	17%
50–59	10	33%
60 or older	2	7%
Sex
Female	23	77%
Male	7	23%
Raceb
Native Hawaiian or Other Pacific Islander	1	3%
Multiple Races	1	3%
Black or African American	1	3%
White	26	87%
Ethnicity
Hispanic or Latino	0	0%
Not Hispanic or Latino	30	100%
Region of Birth
United States	30	100%
Education
Bachelor’s Degree	7	23%
Master’s Degree	12	40%
Doctoral Degree	10	33%
Other	1	3%
Years’ Experience
0–2	4	13%
3–5	8	27%
5–10	8	27%
10 or more	10	33%
Reviews Sensitive Studiesb
Personal Health Information (PHI)	27	90%
Sensitive non-PHI data	28	93%
Populations Reviewedb
Healthy Individuals	29	97%
Patients	22	73%
Children	25	83%
Pregnant Women	21	70%
Older Adults	23	77%
Individuals with sensitive diagnoses	24	80%
Economically disadvantaged individuals	26	87%
Individuals with cognitive impairment	21	70%
Prisoners	22	73%
Other	13	43%
Has Reviewed Data Sharing Plans
Yes	14	47%
No	16	53%

^aNumbers may not add up to 100% due to rounding

^bParticipants were asked to check all that apply so percentages will not equal 100%

Similar to researchers, some IRB members had never heard of a qualitative repository, reporting they ‘don’t know anything’ or ‘hadn’t thought about it before’ when asked what they knew about depositing qualitative data with a repository. Others were slightly familiar with QDS but did not know details of how it operated, as this IRB member stated:

I don’t really know a lot about any repositories of qualitative data at this time, but it is something that’s come on my radar recently that I want to look into more

[IRB 13].

Another IRB member stated:

I don’t know much about it other than we have study teams that request to do it from time to time but as for what that process entails exactly, I don’t know the mechanics of it

[IRB 11].

IRB members and staff were more familiar with quantitative, rather than qualitative, data sharing plans and noted that regulations and guidelines reflected this. This IRB member commented, ‘I think the regulations are probably more built with quantitative in mind’ [IRB 21].

De-identified Data

IRB members generally assumed that only de-identified qualitative data could be shared similar to how de-identified quantitative data is commonly shared. According to this IRB member, ‘I don’t have any problems with it if the data is stripped of identifiers, and participants are aware that it’s going to be shared’ [IRB 13]. This individual said ‘We’re gonna make sure that it’s deidentified before we release it to anybody’ [IRB 1]. However, the current standards for de-identification of quantitative health data—removal of the 18 HIPAA Safe Harbor direct identifiers—will not suffice to anonymize qualitative data—that is, to ensure that participants could not be deductively identified (Meyer, 2018).

Most IRB members were not familiar with anonymizing qualitative data sets. They were also unsure when data might be considered de-identified. This IRB member noted the difficulty of determining when qualitative data are considered de-identified, ‘…unless again the data’s completely de-identified, and that’s the tricky part with the qualitative data’ [IRB 17]. Another IRB member described the competing ideas of what is classified as de-identified data:

What does de-identified mean?… My experience is that the word de-identified is tossed around fairly haphazardly and no one really knows what that means. There is a HIPAA security rule that goes into de-identified, but most qualitative data isn’t covered under HIPAA, so what does that mean in a non-HIPAA center?

[IRB 19].

IRB members expressed uncertainty about what constitutes sensitive data or vulnerable populations, as this person stated:

We don’t have a national, or even an international, set of standards on depositing qualitative data. I think that some of the competing standards would be everything from how do we classify materials as highly vulnerable populations, highly sensitive, all the way to not sensitive, not vulnerable populations

[IRB 2].

In contrast, a few speculated that identifiable data can be shared, as long as the consent form discloses this, ‘…if participants were told from the beginning that their identifiable data will be shared with other researchers, that I believe it’s okay’ [IRB 4]. Interestingly, participants did not discuss the notion of ‘a limited dataset;’ such datasets are commonly shared without participant consent even though they include some HIPAA identifiers that could, in principle, be used to re-identify participants; secondary users typically must enter into data use agreements that require the protection of data confidentiality and prohibit attempts to re-identify participants (2017).

Competing Regulations: We Used to Tell People to Destroy Data

We asked participants to think of competing regulations that could be barriers to QDS. Most described a historical or current tendency of IRBs to ask that data be destroyed, or at least identifiers are destroyed, and that this could conflict with the ability to share data. Most IRB members and staff advised against indicating that data will be destroyed in consent forms but recognized that many people have been given this advice in the past or that different agencies have different standards. They suggested including broad language in consent forms to enable data sharing.

This IRB member noted:

For years and years…researchers would always write into the consent forms that they would destroy certain data, or at least they’d destroy identifiers. But then unfortunately, we discovered that actually runs up against state records retention policies here in

[State X] [IRB 16].

Some IRB members described a policy of advising researchers to destroy identifiers after three years, although neither the NSF, NIH, nor the Common Rule require data destruction (Meyer, 2018). According to this individual:

What we require here is destroy the identifiers and the consent forms after 36 months…And, you know, if they’ve got a requirement to share it, then obviously there’s gonna be some confusion on their end

[IRB 28].

Another IRB member described a need to create new norms given the historical tendency of social scientists to destroy data as a way to protect privacy and confidentiality:

I think that’s a standard…in social science to try to reassure participants in research that their data isn’t gonna be forever available…there’s gonna have to be some clarification on what the norm is

[IRB 7].

For some, the interview itself led IRB members to consider issues they had not considered before about QDS. This IRB member wondered:

We’ll often tell people you have to keep these things for three years. But when I think about it, we probably should be saying…de-identify it and keep it forever, if you want

[IRB 25].

Given the historical tendency of IRBs to advise data destruction, it is unsurprising that consent forms frequently do not address data sharing. We asked IRB members if their IRB had a policy on data sharing when consent forms are silent on the topic, and their answers varied widely.⁵ In some cases, there was no policy, and again our interview was a stimulus for thinking about this issue for the first time. As this IRB member commented, ‘It’s a good question. I don’t think we have a set policy’ [IRB23]. According to this IRB member:

I wish we had one [referring to a policy]. You know, other than that HIPAA authorization language, which specifies generally who will receive what information, we are not great at explaining that in consent forms

[IRB 20].

According to this IRB member, ‘The last five and half months we’ve mandated the use of a consent form that does include data-sharing provisions’ [IRB 28].

Some felt a consent form that was silent on the matter of data sharing provided more flexibility with regard to allowing data sharing compared to a consent form that explicitly says data will not be shared: ‘If it was silent, we…tend to be a little bit more flexible and allowable’ [IRB 12]. The assumption is that sharing de-identified data might be allowable when consent forms are silent on the matter, but what constitutes de-identified qualitative data is still not resolved as this IRB member stated:

…It’s a tricky judgment call sometimes if it’s long transcripts of information then we might say, no, we want you to go back and get consent for that potentially

[IRB 17].

In contrast, others reported that if a consent form was silent, then data sharing was not permitted, ‘If it’s silent on data sharing, then it’s not sharing data. They don’t have approval to do so’ [IRB 13]. Again, this position is contrary to regulations that permit the sharing of ‘limited data sets’ that contain PHI without patient consent (HIPAA 2017).

Impact on IRB Approval Status

When asked what impact data sharing plans might have on IRB approval status, the answers varied widely from completely unsure to no changes, to possible changes including potentially enhancing the application, but no IRB member or staff indicated that research would not be approved. Some suggested data sharing plans would increase the likelihood of approval because it would clarify and transparently let the IRB know about these plans. As this individual stated, ‘We would leap for joy—cuz we don’t normally get something as well thought out as a data-sharing plan’ [IRB 19]. Another individual stated, ‘I think it would make the approvals much, much quicker’ [IRB 20]. Others noted that as long as the consent form stated plans to share data, they were likely to approve the data sharing plans. However, given the lack of standards for de-identifying qualitative data, it is unclear what such a review would accomplish. In contrast, others thought QDS would raise the level of scrutiny by an IRB, potentially making a project ineligible for exempt status, as this person stated:

…If you’re also sharing it, that’s an extra layer of concern that I don’t think meets the exemption category I know some chairs would regard data-sharing as more than a minor thing on research studies and want to take some of them to the full board…

[IRB 28].

Responsibility and Preparedness to Advise Researchers

When asked how prepared they felt to advise researchers, most IRB members and staff reported they are not prepared because they lack experience or that data sharing is under the purview of another official or department (e.g., privacy or compliance officers, legal counsel, computing or technology offices) and required a collaborative effort between various officials. IRB members and staff described a lack of guidance and standards with regards to QDS. According to one individual:

There’s not really a set of standards in terms of institutions, privacy officers, IRBs. I mean, this stuff is all over the place and … people just don’t understand it very well, and there aren’t good answers to give sometimes because we don’t have information

[IRB 20].

This left many IRB members feeling unprepared to advise researchers on legal and regulatory matters.

When asked who was responsible for providing guidance to researchers on legal and regulatory matters pertaining to QDS, most IRB members indicated a collaborative effort that required input from both the home institution and the repository. This individual said:

I think the institution should have relationships with data repositories. And then, manage that relationship with their researchers, and not just leave it up to a researcher-data repository relationship

[IRB 08].

IRB members expressed concerns regarding variations in policies between the institution, the repository, and state laws. According to this IRB member:

The more the data repository can provide that information and…even advise on what they know of state laws, other policies that might apply, the better. But…we do our due diligence and make sure that we don’t see any other rules that might affect us specifically. So it’d be a team effort

[IRB 17].

In contrast, some thought the home institutions were responsible for guiding researchers:

The home institutions need to be able to guide their researchers about this. I would hope that they would understand what the repositories were doing, and offering, and providing

[IRB 7].

Others thought repositories were primarily responsible: ‘Data repository needs to have all of this in order…I would put it more on the…onus of the data repository than the institution’ [IRB 25].

Most often IRB members wondered or speculated about who was responsible, but they were not sure or had not thought about this before. According to this IRB member, ‘That’s a very good question. I don’t know…I honestly haven’t thought much about that.’ [IRB 15]. Another IRB member stated:

I would think the repository’s role is just to hold it, and whatever entity is wanting to gather that, I believe the agreement then would be between the home institution and whoever the next institution is. I may be inaccurate about that

[IRB 24].

According to yet another individual:

So, you know theoretically, I think the right answer is the repository if the repository can have a handle on all of the state and local requirements

[IRB 20].

What is clear from IRB members’ answers is that they are not describing current practices but speculating about what they think would happen, demonstrating a lack of experience with QDS.

Not All Repositories are Created Equally

Curators are highly supportive of data sharing, which is not surprising since they work at data repositories. However, not all curators had experience curating qualitative data sets, let alone sensitive data sets. Only 14/30 curators had personally curated a qualitative data set (Table III). While all curators worked at repositories that could accept qualitative data, some repositories had never actually received a qualitative data set. Others were only capable of accepting data that could be made available open access, prohibiting them from accepting sensitive qualitative data. This curator stated, ‘We’re not keeping that data if it’s sensitive right now’ [C7]. Storing only non-sensitive data helps repositories avoid the challenges of anonymization and maintaining confidentiality, but it limits the number of repositories that qualitative researchers can use if they have sensitive data. As this curator noted, ‘Either we can take it because it doesn’t have sensitive data, or we can’t take it because it does, and, frankly, that’s very unsatisfying’ [C28]. According to this curator:

Table III:

Curators (n=30)

Demographic	Frequency	Percenta
Age
20–29	1	3%
30–39	12	40%
40–49	9	30%
50–59	6	20%
60 or older	2	7%
Sex
Female	14	47%
Male	16	53%
Raceb
Black or African American	2	7%
White	26	86%
Prefer Not to Answer	2	7%
Ethnicity
Hispanic or Latino	1	3%
Not Hispanic or Latino	27	90%
Prefer Not to Answer	2	7%
Region of Birth
United States	25	84%
European Union	4	13%
Oceania	1	3%
Education
Some College	1	3%
Bachelor’s Degree	1	3%
Master’s Degree	14	47%
Doctoral Degree	14	47%
Years’ Experience
0–2	8	27%
3–5	8	27%
5–10	4	13%
10 or more	10	33%
Personally Curates Data
Yes	14	47%
No	16	53%
Curates Sensitive Datab
Personal Health Information (PHI)	12	40%
Sensitive non-PHI Data	16	53%
Types of Data Personally Curatedb
Healthy Individuals	16	53%
Patients	11	36%
Children	6	20%
Pregnant Women	5	17%
Older Adults	10	33%
Individuals with sensitive diagnoses	6	20%
Economically disadvantaged individuals	9	30%
Individuals with cognitive impairment	4	13%
Prisoners	4	13%
Other	13	43%
Types of Data Deposited in Repositoryb
Quantitative	24	80%
Qualitative	23	77%
Mixed Methods	25	83%
Other	6	20%
Types of Data Curated by Repositoryb
None	2	7%
Written Transcripts	17	57%
Audio-recordings	9	30%
Video-recordings	9	30%
Field Notes	12	40%
Archival Data	17	57%
Journals written by participants	5	17%
Other	9	30%

^aNumbers may not add up to 100% due to rounding

^bParticipants were asked to check all that apply so percentages will not equal 100%

We will point you to curators at [another repository] who can do that for you … so that’s kind of where we were at…super-cautious, not really accepting anything that was a human-subjects base

[C26].

Some repositories only accept archival materials such as oral histories where the individual agrees to be identified, or supporting materials such as codebooks, interview guides, blank consent documents, summaries of project findings, or published articles that have been annotated for transparent inquiry, but not the human subject data itself (Moravcsik, 2013).

In contrast, curators at the few repositories who were more experienced with QDS more clearly articulated an anonymization process and confidentiality protections through different layers of access to qualitative data. These curators described reviewing qualitative data sets carefully before deposit:

We review all data before it’s deposited, so no transcript of an interview goes in the repository without at least one—typically two curators having read through it and checked for both direct and indirect identifiers…And we actually find that we probably, in three out of four cases, that we’ve sent transcripts back to depositors and asked for additional changes typically to indirect identifiers but sometimes even direct identifiers that slip through…that’s all during the preparation process of the initial deposit

[C6].

Another curator described the process as follows:

Every piece of information that involves human study subjects is thoroughly scrutinized by a group of people, including members of the institutional review board, the researchers in question, as well as an outside group of individuals within the university…

[C10].

Curators who have more experience with QDS also noted that qualitative data is often only available via restricted access rather than open access in order to protect confidentiality. This curator noted:

We have multiple layers of access and permissions before researchers can actually get at much of this data’ [C1]. Another curator noted, ‘We go through, with all our data but particularly qualitative data to make sure there’s nothing identifiable from any in those data. Many times, we can’t make those data publicly available anyway, so they have to be behind the restricted access barriers

[C18].

Those curators who were more experienced with accepting and anonymizing qualitative data sets were confident in their ability to protect confidentiality:

I’m very confident. We take it very seriously, and we have a number of measures in place from depositing content, which is in a secure environment, to the people who work with the data here internally, our curation staff. They’re trained. They also work in a very secure environment—to the application process for reusing the data, to the environment in which people reuse the data.

[C20].

Given the inability of many repositories to handle sensitive qualitative data, very few curators reported having qualitative-specific guidelines for depositing data. Curators working at repositories that could not accept sensitive data did not necessarily perceive important differences between quantitative and qualitative data given that they could only accept open access data. This curator said, ‘We don’t specify anything differently for qualitative versus quantitative. I think the same rules would apply’ [C2]. According to this curator, ‘There are rules just for all data” [C4]. Another curator stated:

…We have a webpage that has some guidelines for preparing, for de-identifying data…Our deposit form for the archive [requires] that data be de-identified. It doesn’t specify qualitative or quantitative since we’ll take any kind of data

[C12].

Some were unsure whether they had qualitative guidelines at all, as this curator says ‘My sense is that we have internal best practices. I don’t know that we have any published procedures’ [C11]. This curator noted, ‘We have a guide I think that has a section about qualitative studies’ [C20].

In contrast, curators at repositories with experience handling sensitive qualitative data reported having qualitative-specific guidelines to guide depositors. This curator noted:

We are dedicated to only qualitative and mixed-method data, so anything that’s purely quantitative is out of scope for us. So all of our guidelines are in fact specific to preparing qualitative data

[C6].

According to another curator:

The [repository name] has got extensive [qualitative] guidance in detail…Every archive has to handle the strategic challenge, basically, of whether or not it’s got the resources to support…a wide range of data types

[C17].

Even among those repositories that were prepared to handle sensitive qualitative data, some curators reported that they had never used the guidelines because they had never actually received a sensitive qualitative data set. As this curator stated:

We have guidelines. They were developed in association with our social science research community. So, we have a nice policy framework for dealing with qualitative data, human subjects, that is sensitive. We’ve never actually needed to use it. No one’s submitted any of that kind of stuff yet

[C21].

It’s the Wild West: There Are No Agreed Upon Best Practices

While curators were the most familiar and comfortable with data sharing among all the stakeholder groups, curators desired more guidance or standards specific to QDS. We created a lack of knowledge code among this stakeholder group to capture any indications of knowledge curators needed regarding legal, regulatory, or logistical issues. One individual commented, ‘We don’t have the tools and the resources to even understand… what is it we’re asking researchers to do in sharing their data’ [C28].

Curators often spoke of the lack of standards or national guidelines that specified best practices for QDS. According to one curator:

A lot of faculty and librarians and other curators, too, rely on…word of mouth in their communities. And we need voices of authority to say, ‘This is the situation,’ or…’This is the best practice

[C26].

Lack of standards led some to worry that there was no QDS oversight. Monitoring compliance is difficult as there are no auditing bodies or standards for QDS. As this curator commented:

There’s very little auditing that’s happening, so there’s a lot of assumptions about how things are happening, but very little auditing of what is actually being done by researchers…the policies are well thought out that exist when they exist, but the problem is that there’s not a lot of eyes on the content of the data that’s actually being shared to determine if the policies have been followed

[C3].

When asked what resources would be helpful to them, many curators spoke of wanting better guidelines to guide QDS. This curator described wanting tangible examples of what sharing sensitive qualitative data involves in practice:

Having some examples that are out there and prominent that we could point to say…This is what we mean when we say, ‘sharing sensitive data.’ This is how it actually took place…What was the workflow? What was the process?…To have some transparent examples to work with…and to help relate what these issues are and how they could actually translate into practice would be really helpful.

[C28]

Curators also noted the challenges caused by variations in institutional, state, and national policies and laws. This curator noted, ‘These things vary by institutions…my university will likely have very different rules regardless of what the professional association or whatever says I should do’ [C25]. According to this curator, ‘On the legal and regulatory side … from country to country and from state to state…things vary’ [C23]. Curators desired guidance on dealing with the variation in practices and lack of standards:

One of the things that I think is a real challenge is that some of these regulations really vary state to state, even county to county and certainly institution to institution. So I wonder if a centralized service could provide the sort of nuanced level of guidance that would be necessary for answering all of these questions

[C30].

Responsibility and Preparedness to Advise Researchers

When asked how prepared they felt to advise researchers on legal and regulatory matters, the majority of curators were either not prepared or were unsure if they had the knowledge needed. Many of the curators, even those who had curated a sensitive qualitative data set, did not feel prepared and mentioned that they would like to be more prepared on legal and regulatory matters pertaining to qualitative data and privacy laws. For instance, this curator who had personally curated a qualitative data set said:

I don’t feel very prepared…that’s one of my goals for this year is to get trained in the laws surrounding sensitive data in particular, especially with human subjects

[C7].

Another curator who had not personally curated a qualitative data set said:

I feel like I could look up information or consult with other specialists on campus, but I don’t feel like I personally have a really good grasp of that right now

[C5].

This individual said, ‘I generally say, ‘Well, let’s ask IRB’…’ [C12].

Curators working at repositories experienced in QDS more clearly articulated the legal and regulatory responsibilities of repositories. This curator stated, ‘Legally, the responsibility remains with the depositors, as per our deposit agreement with them’ [C23]. According to another curator:

We’re pretty careful when we talk to researchers about our repository to be clear about the function of it and who would have access. But it ultimately is up to the researcher to ensure that they’re adequately protecting the subjects of their study. And the repository itself is not—is not protecting your subjects

[C30].

According to this curator:

We would generally have a policy of not advising people on legal matters. It’s up to them to find their own legal advice if they have those concern. We can only advise them on what our legal obligations are, and what our own limitations would be. But it’s kind of the policy we have not to act as legal advisors ourselves

[C13].

As this curator noted:

I’m not supposed to provide them with legal advice. I can give them literature. I can suggest that they strive to get legal advice, but I am not qualified to give legal advice, and I would not attempt to do so

[C27].

Discussion

Although we used sample sizes that are generally considered adequate for qualitative research studies (N=90 with 30 from each relatively homogenous stakeholder group), we cannot assume that our findings can be generalized to the larger populations of qualitative researchers, IRB members, and data curators. Nevertheless, as an exploratory study, the project points to important and concerning features of today’s research environment, in which data sharing is typically strongly encouraged and sometimes required.

Imagine for a moment that tomorrow the NIH explicitly states that their data sharing policy applies equally to qualitative data and that qualitative researchers with NIH funding are required to share data. Would we be ready?

Our results suggest that we are not ready to share qualitative data due to a lack of experience with, and guidance on, QDS among all stakeholder groups. Researchers are the least knowledgeable about QDS. Many were not aware that qualitative data repositories existed let alone that some repositories are capable of archiving and providing restrictions on who accesses sensitive qualitative data. While some qualitative researchers expressed willingness to share their data, the vast majority do not know how to share data responsibly or requirements of depositing qualitative data. This is particularly concerning given that many IRB members feel ill-prepared to advise researchers on QDS, and data curators feel that researchers have the obligation to protect their data and navigate legal and regulatory matters. Many researchers also conveyed attitudinal barriers to the endeavor as a whole. Future research should also gather evidence of benefits arising from data sharing, such as increased citations or secondary analyses, to increase researcher willingness.

The majority of IRB members, who many researchers would turn to for guidance, also lacked knowledge of how qualitative repositories operated or the protections they could offer, how to de-identify qualitative data, or who was responsible for ensuring legal and regulatory compliance. They expressed needing guidance for QDS and were unprepared to guide researchers about most aspects related to QDS. IRB members frequently hoped repositories could provide guidance or indicated they would seek advice from other institutional officials like privacy officers or legal counsel. We strongly recommend that IRB offices identify the appropriate institutional officials to advise on, and facilitate, QDS, so they can provide researchers with timely referrals when the IRB does not address QDS.

IRBs have widely varied policies regarding informed consent forms and data sharing permissions. This variation in IRB interpretations of federal research regulations is consistent with the findings of earlier studies (Abbott and Grady, 2011). The most incompatible language, which some consent forms still use, is the commitment to destroy data after a certain amount of time or explicit statements that data will not be shared. As a result, many qualitative data sets that have already been collected may not be shareable. In order to avoid this type of restriction going forward, it is essential that researchers and IRB members consider data sharing from the outset and seek consent for data sharing when possible (Meyer, 2018, DuBois et al., 2018). Qualitative researchers and IRB members will need template language and guidance on the informed consent process and incorporating language about data sharing and reuse.

IRB members generally assumed that only de-identified qualitative data can be shared. While removal of the 18 HIPAA Safe Harbor direct identifiers is relatively straightforward, removing indirect identifiers is far more complicated, and there are currently no adequate guidelines or tools for de-identifying qualitative data. Standards and tools for de-identifying sensitive qualitative data are urgently needed.

At the same time, the majority of repositories are not equipped to handle sensitive qualitative data, lack specific guidelines, and will therefore not be suitable for most qualitative researchers who collect data from human subjects. Select experienced repositories have guidelines for sharing sensitive qualitative data and were confident about their repository’s ability to protect confidentiality and provide restricted access. IRB members and researchers lacked knowledge of these options because they were unfamiliar with repositories in general. Curators at these repositories articulated more clearly that they were not responsible for legal and regulatory compliance which they presumed came from researchers and their institutions or IRBs. This leaves us in a challenging situation with both IRBs and repositories desiring better guidance and neither feeling adequately prepared to advise researchers on legal and regulatory matters. It might be helpful if data curators—even if reluctant to provide legal advice—provided researchers and institutions with examples of how sensitive, human subjects data have been shared (e.g., as limited data sets that require data use agreements).

Ethical and productive QDS will require overcoming barriers, creating standards, and changing long held practices among all stakeholder groups. In the next phase of our project, we will develop tools and resources to support researchers as they begin to enter the uncharted territory of QDS. We are developing a software to assist qualitative researchers in anonymizing qualitative data and guidelines on sharing qualitative data for all stakeholders. Specifically, our software will provide the most comprehensive electronic method to date to search for, flag, and enable replacement of direct and indirect identifiers such as profession, setting, institution, or race. The software will support researchers but still requires their input and review before a determination can be made about sharing and access controls. We will then recruit 30 qualitative researchers to pilot our anonymization software and actually deposit their anonymized data for archiving with our project partners at the ICPSR. Our team will also develop a toolkit consisting of the finalized anonymization support software along with guidelines for sharing qualitative data that can be used by all stakeholders. We will disseminate the toolkit to stakeholder groups while evaluating the adoption of QDS practices. By providing materials to support QDS, our goal is to facilitate QDS in an ethical manner and ensure that data are discoverable and usable by others. We do not believe that all qualitative data are appropriate for sharing (e.g., sensitive video footage that cannot be adequately anonymized, sensitive data that participants were told would not be shared, data that would place participants at serious risk of harm). At the same time, there are many advantages to QDS, and many datasets could be shared in a responsible manner—with adequate preparation of the relevant stakeholder groups.