Open data sharing: what could possibly go wrong?

The drive toward open clinical trial data sharing promises to accelerate scientific discovery,⁹ yet shared datasets often lack the context needed for accurate interpretation. Although the National Institutes of Health (NIH) and international research organizations champion data sharing, incomplete documentation creates measurable downstream impacts. A systematic solution must ensure that shared datasets include comprehensive documentation of their nuances and limitations.⁶

Complex trial data stripped of critical context poses significant challenges for secondary analysis. Randomized trials comparing multiple interventions often contain specific design features precluding certain types of analyses. Only 30% of shared trial datasets include sufficient metadata for replication.¹⁰ The scope extends beyond documentation issues. In a striking example, 70 independent teams analyzing an identical neuroimaging dataset reached widely divergent conclusions.⁷ This crucial context is even more likely to be lost in translation when data dictionaries are created as an afterthought following study completion. This challenge grows as automated analyses of public datasets increase, raising the risk of misapplied methods.⁸

This is no longer theoretical, in 1 study, 40% of shared trial datasets have been analyzed in ways that violated their primary design constraints.¹⁹ Clinical trial data labeled as “open” but lacking interoperability led to misinterpretation in 15% of secondary analyses due to missing context.²⁴ Although scientific communities eventually correct errors through letters to editors or retraction requests, retracted articles persist as cited valid evidence long after retraction.¹² This problem extends to journal policies as well. In a recent review of data sharing policies across health research globally, data sharing was required by only 19% (52/273) of health sciences journals, with widely varying conditions for implementation.²³

Clinical trial datasets embody complex development processes shaped by specific aims, design decisions, and operational realities. These elements determine valid data inferences. Original study teams understand these nuances, yet data dictionaries rarely capture these essential constraints after study completion.²¹ This documentation gap grows particularly problematic as repositories accumulate datasets for secondary use.

Truly interoperable clinical trial datasets hold immense potential. Large data collections aim to create FAIR (Findable, Accessible, Interoperable, and Reproducible) resources accelerating discovery through secondary analysis.²⁵ Poor curation and insufficient guidance about dataset strengths and vulnerabilities jeopardize this goal. Analysis shows 12% of shared radiology datasets led to misinterpretation from lack of context, although proper documentation enabled benefits from verification and meta-analyses that outweighed these risks.¹⁸

Our NIH-funded BACPAC BEST study illustrates these challenges. This multisite trial uses Sequential Multiple Assignment Randomized Trial (SMART) design to identify phenotypic characteristics of patients with chronic low back pain responding to common treatments.¹⁵ We specifically selected treatments in different treatment classes (eg, physical therapy, cognitive behavioral therapy, duloxetine) ensuring each intervention contained only single-discipline content (ie, interventions that were purely from 1 treatment modality without combining elements from multiple approaches). This design choice, crucial for precision medicine inferences, explicitly prevents using data for treatment comparative effectiveness. Without proper documentation, analysts might misuse BEST data for inappropriate treatment comparisons that could mislead clinical practice.

Different analytical approaches legitimately yield varying results from identical datasets.¹³ This acceptable variation differs fundamentally from analyses violating core design principles. Better documentation and standards,⁵ not access restrictions, offer solutions. Studies show improved documentation and structured environments reduced misinterpretation by 35% while maintaining utility.²²

Addressing these challenges demands multiple approaches. Community standards must expand beyond basic codebooks and curation. NIH funding should support technical and clinical guides capturing trial complexity. Data dictionaries must evolve into comprehensive documentation including design constraints, analytical boundaries, and key assumptions.¹⁶ Current guidelines like CONSORT, SPIRIT, and CDISC provide valuable frameworks, yet none fully addresses preparing trial data for sophisticated secondary use. FAIR principles offer high-level guidance, but implementation requires expertise and resources many research teams lack.²⁵

Open science and data sharing in clinical trials present opportunities and challenges.^1,3 Sensitive health data shared with proper protocols show reidentification risks as low as 1.2% after anonymization.²⁰ These technical protections, however, are only 1 aspect of data governance. The administrative aspects also present challenges. Manual data access committees have proven ineffective, with evaluations of clinical data warehouses revealing inconsistent governance and only 8% having transparent approval criteria.¹⁷ Furthermore, “data available on request” policies result in less than 20% actual sharing rates due to administrative bottlenecks.¹¹ Although manual oversight shows clear limitations, structured governance approaches like federated analysis offer more promising solutions.²²

We propose unifying existing principles into practical guidance for trial data documentation (Fig. 1). This framework must address technical interoperability and contextual knowledge critical for valid secondary analyses.⁴ Sustained funding support for data preparation and stewardship remains essential. Robust documentation standards, sustained curation funding, and maintained scientific oversight can realize open science's promise while protecting research integrity. These datasets increasingly guide clinical practice and research directions, demanding infrastructure matching their complexity.¹⁴ The scientific community has a responsibility to make data sharing not just possible, but meaningful and reliable.²

Figure 1.

Framework for clinical trial data curation and sharing with 6 essential components that preserve scientific integrity while enabling secondary analysis. Data Quality and Validation: CDISC (Clinical Data Interchange Standards Consortium) SDTM/ADaM (Study Data Tabulation Model/Analysis Data Model) standards and FDA (Food and Drug Administration) quality guidelines. Metadata and Documentation: CONSORT (Consolidated Standards of Reporting Trials) extensions, NIH Pain Common Data Elements, and comprehensive data dictionaries documenting analytical constraints. Data Harmonization and Standardization: PCORnet (Patient-Centered Outcomes Research Network) and OMOP (Observational Medical Outcomes Partnership) common data models. Data Provenance and Attribution: DataCite identifiers, ORCID (Open Researcher and Contributor ID) integration, and W3C (World Wide Web Consortium) PROV standards. Data Preservation and Accessibility: DataVerse repositories, Zenodo archiving, and DataMed indexing services. Training and Support: FORCE11 (Future of Research Communications and e-Scholarship) FAIR (Findable, Accessible, Interoperable, and Reusable) data training and NIH Data Science initiatives.

Conflict of interest statement

The authors have no conflicts of interest to declare.