COVID-19 interventional trials: Analysis of data sharing intentions during a time of pandemic

Abstract

Background

This survey of COVID-19 interventional studies encompasses, and expands upon, a previous publication [1] examining individual participant level data (IPD) sharing intentions for COVID-related trials and publications prior to June 30, 2020.

Methods

Replicating our inclusion criteria from the original survey, we evaluated a larger dataset of 2759 trials and 281 publications in this follow-up survey for willingness to share IPD and studied if sharing sentiment has evolved since the beginning of the pandemic.

Results

We found that 18 months into the pandemic, data sharing intentions remained static at 15% for trials registered through ClinicalTrials.gov (ClinicalTrials.gov is a digital registry of information about publicly and privately funded clinical studies in which human volunteers participate in interventional or observational scientific research) prior to September 19, 2021 compared to our initial survey. However, a comparison of declared intentions to share IPD at the time of publication revealed a noticeable shift: affirmative intentions grew from 21.4% (6/28) in our original publications survey to 57% (160/281) in this survey. Within the subset of studies published within journals affiliated with the International Committee of Medical Journal Editors (ICMJE), positive sharing intentions are even higher (65%).

Conclusions

Although intent to share data at the time of registration has not changed from our prior study in June 2020, there is growing commitment to sharing data reflected in the increasing number of affirmative declarations at the time of publication. Actual sharing of data will accelerate new insights into COVID-19 through secondary re-use of data.

1. Introduction

Individual participant-level data (IPD) is the clinical human subject data that underlies the results of research trials. Voluntary sharing of this data throughout the scientific research community can promote efficient scientific inquiry and expand scientific knowledge. In the urgent case of COVID-19, sharing IPD data efficiently and broadly offers the possibility of accelerated identification of effective or ineffective therapies. In recent years, there has been an increased call to provide optimal access to trial data, specifically IPD, out of an ethical obligation because trial participants have placed themselves at risk. In addition, reproducibility, transparency and increasing research efficiency are also important goals for data sharing [3]. The ICMJE has been particularly influential in this arena. As of July 1, 2018, all manuscripts based on clinical trials must be submitted to ICMJE with a Data Sharing Statement to merit consideration for publication. Furthermore, the registration filing of the underlying clinical trial, if enrolling on or after January 1, 2019, must also include a data sharing plan. These statements must indicate whether data would be shared, describe what and how it would be shared, and provide information about when and how long the data would be available [2]. We embarked on a survey to probe the current climate of data sharing with specificity to COVID-19 trials during a pandemic and to answer 3 key questions. These questions focused on the following: (1) Has the data sharing landscape (intentionality to share) shifted since the beginning of the pandemic for COVID-19 trials as measured by data sharing registration statements? (2) Does intentionality evolve from the time of registration to publication? (3) How successful was the ICMJE data sharing policy in spurring sharing? (Do those trials that published in ICMJE journals say “yes” more often compared to those not published in ICMJE journals?)

2. Methods

We analyzed data sharing intentions in COVID-19 interventional studies as expressed in two separate datasets: ClinicalTrials.gov registered trials and PubMed publications. More than one year since our initial survey [1], there has been a proliferation of activity with a tripling of the number of trials (2759) and a ten-fold increase in the number of publications (281) that met our criteria for inclusion in this follow-up survey. Below we describe the methodology for evaluating (1) ClinicalTrials.gov registration declarations, and (2) PubMed publication data sharing statement extractions. For each unique trial included in the PubMed dataset (2), we also cross-checked for any corresponding trial registrations in (1) and examined the data sharing responses for fidelity. The objective of cross-checking was to answer the following question: If a trial intended to share IPD in the registration phase, was the data actually shared when the trial was published as measured in the data sharing statement?

We conducted a search of ClinicalTrials.gov for COVID-19 interventional trials and identified 2759 relevant studies registered prior to September 19, 2021. Within this dataset, we examined the responses to all ClinicalTrials.gov fields pertaining to intent to share IPD.

The ClinicalTrials.gov dataset comprised all interventional COVID-19 trials registered prior to September 19, 2021 with “COVID” cited either in the title or in the condition field. For these trials, we analyzed the registration information included in the ClinicalTrials.gov data sharing section (Element 12) which includes the “Plan to Share IPD,” “IPD Sharing Plan Description,” "IPD Sharing Time Frame,” and “IPD Sharing URL.” (IPD Data Sharing Description and IPD Data Sharing Timeframe fields contained free-form text entries. These served to expand upon and/or clarify sharing practices.)

We also searched PubMed and identified 545 trials of which 281 met our eligibility criteria. We searched PubMed on August 8, 2021 using a combination of subject headings and keywords1 for COVID-19 and clinical trials. Independent reviewers (RL and ML) screened the publications against predetermined eligibility criteria (included were all COVID-19-related interventional trials in humans including trial results and protocols; excluded were primary systematic reviews, case reports, trials without an intervention, trials where the primary condition was not COVID) and differences were adjudicated by a third independent reviewer (IS). For the resulting interventional human COVID-19 trial publications (281), we conducted an analysis of explicit data sharing statements.

2.1. ClinicalTrials.gov registered trial results

Of 2759 filings, 2046 recorded IPD Sharing Statements. The remainder (713) were Silent about IPD sharing. 1366 of the trials declared a Negative Intention, 417 registered a Positive Intention, and 263 reported being Undecided on the matter (Fig. 1 ).

Fig. 1.

ClinicalTrials.gov IPD data sharing statement of 2759 registered trials.

A subsequent examination of the responses within IPD Sharing Description and the IPD Sharing Timeframe fields revealed occasional discrepancies between the IPD Sharing Statement entry and the trial's actual sharing intention. 74 trials were reclassified based on this analysis (Table 1 ).

Table 1.

Reclassification of Clinicaltrials.gov registered trials based on actual intent.

31 trials were reclassified from Negative Intention to Positive Intention after examination of the “IPD Sharing Plan Description” field, and 11 trials were switched from Undecided to Positive Intention. For some of these trials, the meaning of IPD Sharing seemed to be misunderstood: Despite a negative IPD Sharing Statement, several trials were reclassified to Positive Intention because descriptive language suggested an actual willingness to share “de-identified” or “anonymized” data. For instance, one trial noted that “identified data” are unavailable but requests could be made for sharing “de-identified data.” Other positive reclassifications were made for trials indicating a willingness to share IPD subject to specific jurisdictional regulations. For instance, one trial noted that “IPD is complicated to share under the European General Data Protection Regulative (GDPR) however, it is an aim of the study to make such data available within these regulations.” A different trial suggested that shared data would “need to conform to Nigeria national data regulations.” Trials were also reclassified from Negative Intention to Positive Intention if they permitted sharing IPD data upon request. In many cases, such permission included a caveat that approval would be conditioned on the “legitimacy” or “scientific merit” or other “justifiable” element of the proposed research for which the IPD was being requested.

Moving in the opposite direction, 26 trials were reclassified from Positive Intention to Negative Intention and one was switched from Undecided to Negative Intention. Reasons for these reclassifications included evidence that sharing was restricted to collaborating researchers as well as evidence that sharing was limited to aggregate results rather than the IPD, or to study documents, (e.g. trial protocol, statistical analysis plan, and/or clinical study report), without sharing the underlying IPD.

For the 417 trials with Positive Intention, information provided in the IPD Sharing Timeframe field and/or in the IPD Sharing Description field revealed that sharing schedules varied from sharing immediately to sharing more than 24 months after publication. Trials with an indication of sharing upon “conclusion of the study” or sharing “at publication” were classified as sharing Immediately and represented 33.6% of the Positive Intention trials. 10.8% of the trials intended to share between 1 and 6 months, 19.7% between 6 and 12 months, 11.27% between 12 and 24 months, and 3.8% after 24 months. 20.9% did not offer any information about the timeframe when data would be shared (Table 2 ).

Table 2.

Timeline for sharing.

Immediately	140 (33.6%)
1 to <6 months	45 (10.8%)
6–12 months	82 (19.7%)
12–24 months	47 (11.27%)
24+ months	16 (3.8%)
Unspecified	87 (20.9%)
Total no. agreeing to share	417

Of the 417 trials with positive IPD Sharing Statements, 26.6% named the open-source platform, included a website link from which the data could be accessed (e.g. clinicalstudydatarequest.com, vivli.org, engagezone.msd.com yoda.yale.edu), or provided specific contact information for the person acting as gatekeeper of the data. 3.4% gave general information referring to data accessibility. 70% provided no information about IPD retrieval (Table 3 ).

Table 3.

Specificity of information regarding access to data for 417 trials with positive intention.

Open-Source Platform, Website, or Contact of Gatekeeper Provided	111 (26.6%)
General Information Provided	14 (3.4%)
No Information Provided	292 (70.0%)
Total	417

2.2. COVID-19 PubMed publication results

281 publications met our criteria for COVID-19 interventional trials in humans - 34 publications described study protocols and the remainder reported trial results. Of the publications we surveyed, 57% included a data sharing statement expressing positive intentions to share, and 10% expressed negative intentions regarding sharing. For 33% of the publications there was no data sharing statement or insufficient information to classify either a positive or negative sharing intention (Fig. 2 ).

Fig. 2.

Sharing intent of 281 studies published before August 9, 2021.

133 of the 281 published trials had ClinicalTrials.gov registrations, 47 trials reported foreign registrations (e.g., EU Clinical Trial Register, Chinese Clinical Trial Registry, Brazilian Registry of Clinical Trials, Iranian Registry of Clinical Trials, Clinical Trial Registry of India), and specific trial registration information was unavailable for the remainder. 147 of the PubMed dataset are studies published in journals affiliated with the International Committee of Medical Journal Editors (ICMJE) and 134 are not. Of the 147 ICMJE publications, 65% (95) have positive sharing statements and 12% (18) have negative statements. 23% (34) had No Statement or Inclusive Information (Fig. 3 ).

Fig. 3.

Sharing intent of the subset of COVID-19 intervention studies (147/281) published in ICMJE Journals.

Of the 133 trials registered on ClinicalTrials.gov, 97 corresponded with trials included in our survey of 2759 Clinicaltrials.gov interventional studies. Of these 97 “Follow-Through Cases,” there was a noticeable shift from either Undecided, Negative or Silent Sharing Intent at the time of registration to an Affirmative Sharing Statement in the resulting publication. Only 9% of the Follow-Through Cases reversed their original intention from Positive or Undecided IPD Sharing to an ultimate unwillingness to share (Table 4 ).

Table 4.

Fidelity of data sharing intent for ClinicalTrials.gov registered trials and their subsequent ICMJE publication data sharing statements.

3. Discussion

3.1. Data sharing declarations in ClinicalTrials.gov registered trial dataset

Element 12, the IPD data sharing component of the ClinicalTrials.gov registration, is an optional filing module seeking input regarding specific intention to share IPD as well as information about how and when the IPD are made available to researchers. Although an elective component of trial registration, designating a specific sharing intention as well as providing information about how and when IPD is accessible is a requirement of the ICMJE, which represents an influential group of medical journals. Therefore, any investigators aspiring to publish their research through journals associated with the ICMJE would be obligated to make either an affirmative sharing declaration with supplemental access information, or a negative sharing declaration.2

Only 2.6% of all 2759 trials identified in ClinicalTrials.gov had a positive intention to share within a year of publication or conclusion and provided clear information about the mechanism for accessing IPD. The Institute of Medicine (IOM) lays out a benchmark of sharing within one year of publication in its Guidelines for Sharing Clinical Trial Data [7]. We believe that this 2.6% does not reflect the spirit of the IOM recommendations. Although ICMJE requires information about sharing, it does not mandate that trials willing to share do so in a timely manner. Since Element 12 in ClinicalTrials.gov is optional at registration, researchers not anticipating publication in ICMJE, or those unaware of ICMJE's policy, might postpone decision-making about IPD Sharing by indicating an Undecided position or might leave the fields blank by default. Likewise, trials that made affirmative sharing declaration in the registration, but shared no timeframe and/or access information, may have been willing to provide more specific information if it were made a mandatory field. This finding, coupled with descriptions of IPD sharing that were inconsistent with the stated intent (Table 2) as well as numerous descriptions that reflected ambiguity regarding the very meaning of the term “IPD Sharing” itself is not unique to our survey and has been reported by others as well [4,5].

3.2. Data sharing statements in the publications dataset

In contrast to the original survey [1] in which 7 out of 28 (25%) included a data sharing statement, 66.5% of the current 281 publications provided a data sharing statement. The more recent publications dataset suggests that there may be a shifting attitude toward the value of IPD sharing: 56.9% of the publications in this survey expressed willingness to share in comparison to 21.4% in the earlier survey. We hope that this result is attributable to an increasing sense of global responsibility among researchers regarding the need to collaboratively address the COVID pandemic. It is also possible that publication bias may have been a contributing factor toward subsetting the population of trials and researchers that are potentially more pre-disposed to sharing positive COVID trial results in public facing high impact journals.

The publications dataset overlaps with the Clinicaltrials.gov registrations dataset for 97 studies. Within this subgroup, 22.7% reversed from their original negative intention at registration to a positive declaration at the time of publication. It is worth noting that the ICMJE implements a strict publishing policy [2] requiring a definitive positive or negative IPD Sharing Statement in the Clinicaltrials.gov filing. However, it does not insist that trials adhere to their originally stated intention upon publication (4.1% of publications switched their IPD intentions from positive at registration to negative at publication). Additionally, one quarter (25) of the subset that were Undecided or Silent about IPD Sharing at registration, subsequently expressed a willingness to share after publication. While non-committal may be an easy default or an innocuous choice to satisfy registration requirements - especially given that Clinicaltrials.gov filings typically are not publicly viewed – upon publication, researchers may understand that their intentions will be broadly revealed and consequently may feel more inclined to make an affirmative declaration. Furthermore, once a trial is concluded and results are prepared for publication, researchers may have more confidence in the value of their data, thereby viewing IPD sharing as a more compelling option.

3.3. Limitations

This survey has the following limitations. First, we analyzed only ClinicalTrials.gov and not other internationally recognized registers such as the EU Clinical Trial Register or registers of the World Health Organization's International Clinical Trials Registry Platform. As this is a follow-on survey, we wished the data to be comparable to our initial survey and ClinicalTrials.gov contains the majority of the world's trial registrations. Additionally, for the publications study we conducted a search in PubMed but did not search additional databases; however, this is considered accepted practice in many COVID-19-related rapid reviews. Many of the journals we included were not from journals that follow ICMJE recommendations, and therefore may not have been required to include data sharing statements. Finally, although we examined the data sharing statements of all publications in the dataset, we can not draw conclusions about whether the IPD would actually be delivered as indicated. Recent research suggests that there may be a gap between publications' declared and actual data sharing [6].

4. Conclusions

We summarize our key recommendations in Box 1 below.

Box 1. Key recommendations.

• •ICMJE journals should monitor for missing data sharing statements
• •
  Clinicaltrials.gov should require IPD sharing intentions (Element 12)

  • o
    Require those with “yes” intention to specify information for obtaining access to IPD
  • o
    Require those with “yes” intention to specify time frame for IPD availability
  • o
    ClinicalTrials.gov to consider re-aligning choices with ICMJE (eliminating the “undecided” selection)
• •The biomedical community should develop increased training or education on the meaning of IPD data sharing and IPD data sharing terminology

Alt-text: Box 1

The low positive percentage of researchers with stated willingness to share data at the time of initial registration may be due to several factors including: thoughts about data sharing are not a priority in early trial stages; researchers have misunderstanding about IPD terminology; and, the optionality of data sharing fields leads to complacency. Education, publicity campaigns and policy changes present strategies and opportunities to overcome these factors. The ICMJE policy requiring data sharing statements seems to have promoted the objective of increasing data sharing in the case of COVID-19, however journals could further increase data sharing with ongoing monitoring of this information to align with the ICMJE policy. The results from this study will hopefully be used to initiate discourse regarding current policies and increase efforts to encourage data sharing. There are now several policies on the horizon to prompt such change as well as an ecosystem of stable IPD repositories [[8], [9], [10]] that exist to serve the community. It is our hope that the culture of IPD data sharing does not hinder scientific advances but rather serves to accelerate and perhaps even lead the way to new science in the way sharing of the SARS-CoV-2 virus' genomic sequence led research in the early days of the epidemic [11].

Contributor information

The authors confirm contribution to the paper as follows:

Study conception, design: RL, IS, ML, MI.

Analysis and interpretation of results: all authors.

Draft manuscript preparation: KL, RL, IS, ML, MI.

Ethics approval and consent to participate:

Not applicable.

Consent for publication

Not applicable.

Availability of data and materials

Following publication, all primary data tables will be available upon request to rli@vivli.org.

Authors' contributions

RL and IS conceived of the original idea for this paper and developed the concepts. RL, IS, ML and KL conducted the analyses, interpreted the data, and drafted the manuscript. SN and MVI sourced the data and provided important design feedback. MVI and FR revised and contributed to the final version. CDA, EG, and DZ assisted in project management and provided important feedback. All authors read and approved the final manuscript.

Funding

No specific funding was obtained for this study.

Declaration of Competing Interest

The authors do not declare any competing interests, or any specific funding tied to this manuscript.