Research Code Sharing in Support of Gold Standard Science

Abstract

Sharing research code in an open access version-controlled repository offers significant benefits for both science as a whole and for individual researchers. In this article, we focus on this practice, which is fully aligned with the NIH’s Gold Standard Science (GSS) program as well as FAIR (findable, accessible, interoperable, reusable) and TRUST (transparency, responsibility, user focus, sustainability, technology) principles. Gold Standard Science supports open science by emphasizing transparency, reproducibility, and the use of best practices that enable others to verify and extend research. Pairing a research article’s cited data snapshot with a versioned, environment-specific code release, deposited in a companion code repository, ensures that, upon submission to a medical journal, readers and reviewers can directly verify results. An executable and updatable companion code repository complements, rather than replaces, established research data repositories. When code underlying medical research results is made openly available, then other scientists can inspect, run, and validate analyses. These activities enhance reproducibility, which is a core aim of GSS. Shared code also facilitates collaborative innovation by allowing researchers to extend the utility of the code to new datasets and applications. For researchers, code sharing can increase visibility, credibility, and citation impact. Demonstrating transparency through shared executable and updatable code builds trust with journal readers, peer reviewers, funders, and peers. Shared code in an open access repository signals adherence to high standards of scientific integrity and attracts opportunities for collaboration. A researcher who shares code receives recognition as a leader in reproducible, trustworthy research consistent with NIH’s GSS principles.

Introduction

In this article, we focus on a practical and important complement to data sharing that is aligned with Gold Standard Science (GSS), FAIR (findable, accessible, interoperable, and reusable), and TRUST (transparency, responsibility, user focus, sustainability, technology): pairing each journal article’s cited data snapshot with a versioned and environment-specific code release, deposited in TRUST-aligned repositories. Through this approach, readers can verify results and build reliably on prior work.

The integrity of medical science relies on a foundation of credibility, reproducibility, and transparency to drive innovation and progress in clinical care. These values, which are broadly understood as TRUST, are long recognized by the scientific and medical communities and are now further articulated in the National Institutes of Health’s (NIH) recently published Plan to Drive Gold-Standard Science. ¹ In that plan, NIH outlines nine tenets of scientific rigor, beginning with reproducibility and transparency.

These recommendations build on over three decades of the open science movement, which strives to make every stage of research (from planning and data collection to analysis, publication, and reuse) transparent, accessible, and reusable by default. ² Early anchors include arXiv (1991) for preprints, ³ the Bermuda Principles (1996-1997) for rapid genomic data release, ⁴ and the Budapest Open Access Initiative (2002) that defined open access and catalyzed policy adoption. ⁵ In the last decade, communities have standardized how to share and preserve research outputs: the FAIR principles (2016) for making data and code findable, accessible, interoperable, and reusable ⁶ ; the TRUST principles (2020) for trustworthy repositories ⁷ ; and FAIR for Research Software (FAIR4RS, 2022) ⁸ and software-citation guidance for code. ⁹ Collectively, they have laid the groundwork for credible, reusable science, making it practical to verify findings and build reliably on prior work.

Principles of Open Access Science

Open science emphasizes transparency, reproducibility, and collaboration to make scientific research results—including publications, data, code, physical samples, and methods—openly accessible to everyone. Three quality frameworks—GSS, FAIR, and TRUST—are examples of open science principles. They are complementary, and each provides guidance for different dimensions of the scientific process (Figure 1). NIH’s GSS sets expectations for how science should be done (eg, rigor, transparency, falsifiability, and bias-mitigating peer review), shaping the behaviors of investigators and institutions. Those behaviors yield the scientific record in three parts: (1) manuscripts (the narrative, claims, and insights), (2) datasets (the evidence), and (3) code and computing environment specifications (the method to regenerate results). The FAIR principles specify the qualities of these objects so others can find, access, interoperate with, and reuse them. The TRUST principles specify that these objects should live in repositories with transparent governance, versioning and durable landing pages, user focus, sustainability, and sound technology to ensure long-term citability and stewardship.

Figure 1.

Three open science frameworks (GSS, FAIR, and Trust) that are complementary to each other and guide different aspects of the scientific process.

Abbreviations: FAIR, findable, accessible, interoperable, and reusable; TRUST, transparency, responsibility, user focus, sustainability, technology.

The 2023 NIH Data Management and Sharing (DMS) Policy codifies parts of this agenda into funder requirements for data: planning, budgeting, and appropriate sharing aligned with privacy, consent, and legal/ethical limits. ² The policy requires an approved DMS Plan at the time of application, requires updates in progress reports, and allows budgeting for data management and sharing activities. ² It directs investigators to maximize appropriate sharing of scientific data, typically via trusted research data repositories that support persistent identifiers, versioning, preservation, and clear access models (including controlled access for human-participant data when needed). ² While the DMS Policy’s primary focus is data, NIH notes that sharing analysis scripts, software, and code is a best practice that strengthens reproducibility; it encourages such sharing but does not mandate public code release. ³

Companion Repositories for Code

The National Institutes of Health does not specify a uniform format for deposited code or that it be made easily sharable. Building on the DMS Policy’s focus on data, code sharing can be improved with standardized practices focused on sharing and reproducibility. For code, there are two practical and complementary approaches: (1) A permanent archive of code releases for citation (eg, archiving versioned releases with unique DOIs) that creates non-mutable, citable snapshots that match a manuscript’s analyses; and (2) An updatable, executable companion code repository for collaboration and reuse where authors can maintain issues, pull requests, and documentation. In the latter setting, code should be open access, meaning that the source is publicly available under a license that permits use, modification, and redistribution. Without an explicit open-source or permissive license, others generally cannot legally modify or redistribute code. ¹⁰ Providing a clear license and archiving versioned releases ensures that readers can both reuse the software and cite the exact version used in the study. ¹¹ An updatable and executable companion repository for code complements, rather than replaces, established research data repositories. The contents of a companion repository are listed in Table 1.

Table 1.

Contents of a Companion Repository for Executable and Updatable Code.

• README File: Plain-English run instructions and repository map; link to the manuscript and to the data repository accession/DOI for the exact data snapshot used. • Analysis code and workflows: Notebooks and scripts (eg, R, Python, Stata/MP syntax exports) and/or workflow files that regenerate all tables, figures, and results without manual edits. • Computational environment: A machine-readable spec so others can run the code (eg, Dockerfile, environment.yml, requirements.txt), and any version/seed settings needed for deterministic output. • Provenance and transformation notes: Stepwise description of data preparation (filters, recodes, exclusions, imputation rules), with a data dictionary, naming conventions, schema and references to any taxonomies/ontologies used. • Public exemplars (optional): Small, non-sensitive sample, aggregate or clearly labeled synthetic files only to demonstrate scripts; the real study data live in a data repository. • Citation and licensing: CITATION.cff (or equivalent) with how to cite the software, a clear LICENSE (OSI-approved where possible), and instructions for submitting issues, pull requests, etc. • Versioning and releases: Tagged releases with notes; archive each release to a DOI-issuing service (eg, Zenodo) for a citable, non-mutable snapshot. • Do not include participant-level human data, secrets, tokens, or other sensitive information in the code repository. Use controlled-access data repositories as appropriate and reference their accession in the README.

Abbreviation: OSI, Open Source Initiative.

The National Institutes of Health encourages the use of collaborative code platforms with long-term sustainability that support web-based development, sharing, integration, issue tracking, and version control. These platforms are often called code repositories or code forges; examples include GitHub, GitLab, and Bitbucket. ¹²

Executable Code in a Companion Repository

In this context, “executable code” means analysis code that a third party can run end-to-end in a documented computational environment. Authors should provide scripts, notebooks, and an environment specification (eg, container, environment.yml, requirements.txt, or renv.lock) so others can rerun the pipeline. If code is provided in publicly available executable, structured formats, then peer reviewers and secondary researchers can validate analyses, rerun workflows, and detect discrepancies. ¹³ Even with access to a complete and/or raw dataset with the code to generate the precise analytic dataset, it is possible for code to be non-executable in an environment other than that of the author because of missing supporting code, errors in code, or intentional sharing of illustrative/teaching code rather than fully runnable scripts. As much as 76% of public notebook code is technically non-executable under strict standards. ¹⁴ A 2022 study by Trisovic et al ¹⁵ of 9000 code files in an open-source data plus code repository platform for sharing, archiving, and citing research data, showed that 74% failed to complete without error in the initial execution, and 56% still failed following code cleaning. For analyses that involve proprietary tools or sensitive code, authors may provide a temporary private link or an access-controlled repository for reviewers to examine the work in an environment that mimics the proprietary context. ¹⁶ Public release of code is encouraged for reproducibility but is not required under the NIH DMS Policy. ¹⁷

Updatable Code in a Companion Repository

At scale, there are two practical pathways to support updatable, executable code alongside frozen datasets. First, some generalist/archival repositories (and certain research data repositories) let authors co-locate a versioned code release with the cited data snapshot, preserving both with persistent identifiers (eg, DOIs) to ensure their long-term availability and integrity as an official record of research artifacts. ¹⁸ Second, data repositories can partner with modern code platforms (eg, via integrations) so that when authors create a formal code release, that exact version is archived and assigned its own DOI for citation and reproducibility. ¹⁹ In both scenarios, each new code release receives a distinct DOI (with links between versions), ensuring that readers can always retrieve the precise code for a specific analysis. Figure 2 illustrates how journals, data repositories, code repositories, and collaborative code platforms interact in the scientific process.

Figure 2.

How journals, data repositories, code repositories, and collaborative code platforms interact to produce research objects and research records leading to dissemination and discovery.

Abbreviations: DOI, Digital Object Identifier; PID, Persistent Identifier.

An updatable companion repository lets code evolve through bug fixes, documentation, and new methods without changing what was cited in the publication. Authors can issue new releases with clear changelogs, while the manuscript continues to reference the original archived release/DOI used for its results. Framed this way, the repository becomes an open, version-controlled, community-maintained record. If code is submitted to a companion updatable repository, then future viewers may extend, update, or recompute the analyses, ensuring that the original work remains transparent and verifiable long after its initial appearance. ²⁰

Benefits of Sharing Research Code in Companion Repositories

Submitting research code in a transparent companion repository with a persistent identifier such as a DOI, and citing or linking it within a journal article, enhances the study’s visibility and overall impact within the research community. ²¹ Code sharing also demonstrates transparency and a commitment to reproducibility, which can bolster the author’s reputation and that of their affiliated institution. ²² Practically, code sharing supports methodological rigor and error detection through verification by peers as well as serving as a foundation for new collaborations. ²³

A citable companion code repository allows others to validate, reproduce, and build upon the work. These actions increase the credibility and trustworthiness of the published results and could lead to higher citation rates and greater recognition for the research, ²⁴ although the topic has not been extensively studied. Furthermore, non-citation quantitative measures of impact, such as forks (new repositories created from existing ones) and downloads, reveal engagement and reuse of code. They can provide meaningful evidence of impact, although they are not represented by citations. ²⁵ COARA (Coalition for Advancing Research Assessment) endorses several qualitative methods, such as peer review and expert opinion, for recognizing the impact of code that goes beyond traditional citation metrics. ²⁶ In the same way that article- and dataset-level metrics have matured (eg, Altmetric attention scores for papers/software, ²⁷ Make Data Count/DataCite usage, ²⁸ and citation tracking), ²⁹ code repositories are on a similar trajectory. As community norms around software citation, versioned DOIs for releases, and standardized usage signals continue to evolve and gain traction, companion code repositories will increasingly serve as recognized, citable scholarly outputs with impact that is measurable alongside traditional citations.

Potential Barriers

Direct costs are typically minimal for public repositories on platforms such as GitHub or GitLab, but indirect costs related to the time and effort burden of documentation, organizing code, environment capture, and maintenance can be substantial. Dedicated funding for these efforts may be needed to secure contributions and ensure sustainability. Researchers who share code may worry that competing groups may use their code without giving them adequate credit, especially if they have not yet published all intended results or completed larger projects that rely on that code. Clear license terms can help define how the code may be used.

The primary goal of most open research code repositories is to make code freely available under recognized open-source licenses for reuse, as well as to collaborate in research and software development under licenses like MIT (Massachusetts Institute of Technology), ³⁰ GNU (GNU’s Not Unix), ³¹ or CC (Creative Commons). ³² Without an explicit open-source license, code is proprietary and not legally reusable. However, practical constraints can override or limit openness. Sharing code in a code repository is subject to privacy and confidentiality protection as well as (1) policies of the funding bodies and their institutions, (2) restrictions and policies set by applicable federal, Tribal, state, and local laws, and (3) existing or anticipated agreements. ¹⁷ In industry-sponsored studies, contracts often constrain code release (eg, intellectual property ownership, embargoes, or review clauses). When sharing is limited, journals can still encourage verification bundles so reviewers can evaluate methods without disclosing proprietary data. Finally, code from industry-sponsored studies is typically not shared, and research contracts often prohibit such sharing. ³³

Conclusions

We expect two reinforcing trends in code sharing over the next few years. First, we anticipate increased encouragement and infrastructure for research code sharing across funders and journals. Many medical researchers—NIH-funded or not—are already archiving code snapshots alongside data deposits. Second, we anticipate that voluntary archiving of code in companion repositories by medical researchers will become more prevalent, which will facilitate greater transparency, reproducibility, and reuse of research software.

We commend NIH for its Gold Standard Science initiative and encouragement of data and code sharing. Evidence from multiple assessments shows persistent reproducibility challenges across parts of biomedicine, and surveys report that many researchers perceive a problem, although estimates vary by field and method.^19-21 Sharing executable, environment-pinned code with a cited data snapshot makes work more transparent and testable, and thus more trustworthy and potentially more impactful. Over time, such contributions will shift the culture of medical science toward one that embodies the rigor envisioned in NIH’s Gold Standard plan.