Abstract
The members of the International Nucleotide Sequence Database Collaboration (INSDC; https://insdc.org) have built systems to collect, archive and disseminate sequence data for more than four decades. The three collaborating organizations, the National Library of Medicine, National Center for Biotechnology Information (NLM-NCBI) in the United States, Research Organization of Information and Systems, National Institute of Genetics (ROIS-NIG) in Japan; and the European Molecular Biology Laboratory-European Bioinformatics Institute (EMBL-EBI) formalized their relationship through the adoption of an arrangement which documents their commitment to free and open access to genomic sequences. The INSDC is committed to expand the collaboration to be more representative of the global community of sequences and users. Diversifying participation through new membership will advance open science and data sharing and, in turn, drive innovation. This expansion will additionally benefit the INSDC and its broad user base by providing additional diverse perspectives as it explores emerging areas of data management, including federation, attribution and management.
Graphical Abstract
Graphical Abstract.
Introduction
The International Nucleotide Sequence Database (INSDC; https://www.insdc.org/) (1) is committed to open science and data sharing. It is one of the oldest and most lauded data sharing collaborations in science. The INSDC captures, preserves, and presents globally comprehensive nucleic acid sequence information as part of the permanent scientific record and as a forum for sharing amongst the broad scientific community. The collaboration is comprised of three members that maintain submission systems for data deposition and analysis and retrieval tools for data acquisition. Data is exchanged daily so that each site has the full complement of data submitted to INSDC. The INSDC has grown significantly and contains 32 million next generation sequence runs and 4.76 billion assembled sequences (Figure 1). These figures do not include controlled access human data which managed separately by each member and not shared between INSDC members.
Figure 1.
Growth of numbers of sequences in the INSDC repositories. Number of assembled sequences (circle; left axis) and next generation sequencing runs (triangles; right axis)
Reflections
As early as 1979, the importance of building archives to house nucleotide sequence data was understood, due to the recognition that large, comprehensive sequence collections were key elements of biological discovery (2). DNA sequencing was costly and digitizing sequence information was manual. In 1982, EMBL Data Library (now ENA) (3) in Europe and GenBank® (4), originally housed at Los Alamos National Laboratory (now at the National Center for Biotechnology Information, National Library of Medicine), partnered through an informal agreement to collect and share nucleotide sequence data globally. In 1986, the DNA Databank of Japan (DDBJ) (5) joined. The three collaborating centers adopted the name International Nucleotide Sequence Database Collaboration (INSDC). This informal collaboration has lasted over 40 years due to the strong relationship between the National Library of Medicine, National Center for Biotechnology Information (NLM-NCBI) in the US, Research Organization of Information and Systems, National Institute of Genetics (ROIS-NIG) in Japan; and the European Molecular Biology Laboratory-European Bioinformatics Institute (EMBL-EBI). In 2023, as sequencing technology has improved and become cheaper, the use of DNA sequencing and genetic sequences have become more ubiquitous across many scientific disciplines and by many more researchers globally, the founding members of INSDC took steps to formalize their collaboration and look toward a future organization with more global representation.
In 2023, the three founding members of INSDC signed a Founders Arrangement that codifies the current collaboration among NLM-NCBI, ROIS-NIG, and EMBL-EBI and documents the longstanding, mutual understanding of the purpose, governance, and foundational principles of the INSDC. The member organizations committed to furthering open science and data sharing by signing the document. The Founders Arrangement established a governance structure which included the formation of two distinct committees, an Executive Committee (EC) and an Implementation Committee (IC). The INSDC EC is responsible for guiding the strategic direction of the INSDC while the IC is responsible for establishing INSDC policies and procedures for building and maintaining nucleotide sequence databases.
A Membership Arrangement was also written and signed by each founding INSDC Member. The Membership Arrangement affirms that INSDC Member institutions are independent government or non-profit organizations that manage nucleotide sequence databases that capture, preserve, and present comprehensive nucleotide sequence information and annotations to preserve the scientific record and enable broad sharing of such data. Members commit to providing free and unrestricted access to data resources and services that are integral to global scientific research and discovery. The Arrangement sets out expectations for all Members which include, collecting data and providing mechanisms for data updates, providing mechanisms for submission and access to submitted data, associating a citable accession number to submitted objects, ensuring submitted data meets minimal standards and is FAIR (10) and secure. When additional organizations meet the criteria for INSDC membership, they will also be required to sign this arrangement.
The Founders recognized that genomic sequencing is now nearly ubiquitous, in many disciplines including the life sciences and public health, as compared to past decades and that the needs of the collaboration and the global scientific community would be better served by diversifying participation through the incorporation of new members into the collaboration. To facilitate this, the INSDC aims to expand its reach by including collaborators that host data repositories for specific global regions or limited data types, as well as, organizations representing all continents covering all organisms and all environments. INSDC expansion will ensure that its operation reflects emerging trends in scientific discovery and societal needs for the benefits that result from sequence information. Additionally, INSDC expansion will facilitate the building of equitable systems that enable the global benefit from sequence information, and to reflect diverse perspectives of genetic sequence generators and managing repositories in its operations.
In 2024, the INSDC website was revised to emphasize these goals of expanding the reach of collaboration and providing transparency into the activities and governance of the INSDC. In addition to technical documentation previously available from the legacy site, the updated site includes two new sections, ‘About INSDC’ and ‘Global Participation’.
The ‘About INSDC’ section describes the INSDC collaboration, Mission and Vision, Governance model, and will include information about the International Stakeholders Committee (ISC), once formed. Additionally, the section presents contact information for the current members and a list of participating databases. The ‘Global Participation’ page describes INSDC’s plan to expand the membership to make the collaboration more diverse and representative of the entire world in alignment with the global production of nucleotide sequence. Links to the Founders Arrangement, Membership Arrangement, and Membership Acceptance and Performance Guidelines can also be found on this page.
The INSDC also introduced two membership levels: Full Members, who build and maintain sequence repositories for all data types supported in the INSDC, and Associate Members, who build and maintain sequence repositories that are limited in scope by region, scientific domain, or data type. For example, an Associate Member may manage a resource that contains only bacterial pathogen sequences, or raw reads from a specific geographical region such as South America. Associate Membership may be a step in the path toward Full Membership or it may be an endpoint. Regardless of membership level, all members must exhibit good data stewardship and adhere to the principles set forth in the Membership Arrangement.
All prospective INSDC Members must demonstrate experience with hosting open nucleotide sequence data archives with submission platforms, processing and validation systems, and mechanisms for public search and retrieval. The initial phase of demonstrating technical capacity includes entering into a data-brokering relationship with one of the current INSDC Members where they collect sequence data from submitters and submit that data to INSDC in a timely manner as a precursor to full data exchange. Prospective members should also start mirroring INSDC data for presentation to their users.
The INSDC has developed a Maturity Model that uses a series of objective criteria for all of the categories in each of five areas listed below that are arranged in increasing levels of maturity. The model is currently available upon request from https://www.insdc.org/contact/. The INSDC anticipates making the Maturity Model fully available via the website after its validation with initial prospective members.
The five areas that prospective members need to demonstrate proficiency in are detailed in the Membership Acceptance and Performance Guidelines:
Governance and Institutional Context
Technical Infrastructure
Data Operations
Communications and Engagement
Quality
When a prospective member demonstrates proficiency in all areas, the EC will recommend them to the IC for a vote. If accepted, the new Member will sign the Membership Arrangement and become an INSDC member and have the same rights and responsibilities as the founding members as described in the Membership Arrangement.
In the coming year, the INSDC EC will establish an ad hoc International Stakeholders Committee (ISC) which will be comprised of a diverse group of external experts who are expected to promote the principles and activities of INSDC and provide input on different stakeholders’ needs.
Current trends
Minimal standards effort
As part of the initiative to formalize the collaboration and provide a mechanism for broader global participation, INSDC undertook an evaluation of the data acceptance criteria, validation rules, and formats used by each member. This work was charged by the EC and carried out by the IC which formed ad hoc working groups to review acceptance criteria and propose pathways to increase consistency and alignment with other standards organizations such as the Genomic Standards Consortium (GSC; https://www.gensc.org/), the Public Health Alliance for Genomic Epidemiology (PHA4GE; https://pha4ge.org/), and the Global Alliance for Genomics and Health (GA4GH; https://www.ga4gh.org/), across eleven categories of data and metadata, including reads, sequences, annotation, samples and projects. The working groups identified areas of divergence and are continuing work to reach consensus. Once minimal consensus standards are approved by the INSDC IC and EC they will be published on the insdc.org website as a guide for new members and data depositors. The INSDC anticipates that implementation will be carried out over several years given the complex nature of INSDC data repositories and services.
Importance of metadata
A nucleotide sequence without any annotation or contextual metadata is not very useful in isolation. The strength of a comprehensive nucleotide sequence repository is that users can compare unknown sequences with the sequences in the database to learn about the source and function of the sequences. However, if the sequence records do not contain any information about when and where the sequenced sample was isolated, its utility decreases. Contextual information about the sequence and about the sample that was sequenced is critical across many disciplines including public health and biodiversity. To facilitate collecting rich contextual metadata, each INSDC member requests information about when and where a sequenced sample was isolated and information about the conditions of collections. To facilitate the collection of this information, packages or checklists have been created that collect appropriate metadata for a particular sample type with a series of mandatory and recommended attributes (6–8). For example, for a human pathogen sample, organism name, strain, collection date, geolocation, and isolation source are mandatory, while information about the host and their disease status are recommended (9). By the end of 2024, all new submissions to INSDC archives require spatiotemporal information (geolocation and collection date) to be included with the sample description to enhance the utility of the sequences.
Future outlook
Its decades of experience as the world's preeminent open access sequence repositories, as well as its recent activities to formalize and expand membership, have helped the INSDC identify several areas of data management that are currently undergoing evolution or emerging as areas in which new development is needed to meet the needs of data submitters and users or to ensure the sustainability of sequence repositories.
Data exchange versus data federation
Historically, the INSDC has exchanged all data among members. This provided redundancy in the case of data loss, synchrony in content, and fast retrieval for users around the world. Despite these benefits, data exchange also introduces challenges to data management. For example, because content can be updated or corrected, checks and validations must be instantiated to ensure all sites have the most recent copies of the data. Furthermore, data exchange is limited to content that is compatible with INSDC-defined formats used for transfer. As the volume of data increases and technology improves, data federation can become a reasonable and cost-effective alternative to data exchange for certain types of sequence data or its associated metadata. If data is federated, it would not be mirrored to all INSDC repositories. Metadata would be exchanged which would contain contextual information about the data and a pointer to the location of the data. As part of the aforementioned minimal standards evaluation, the INSDC is also reviewing the requirements for the availability of each data type. This activity will identify those data types for which exchange is deemed essential and those for which the concept of data federation may be suitable. These categorizations will inform future plans for technical elements of data management and expectations for future members of the INSDC.
Best practices for data attribution
Open access is a core tenet of the INSDC and is highly aligned with the FAIR principles for data reuse (10). No restrictions are placed on the use of data made available through INSDC databases. However, the INSDC recognizes some groups may choose to withhold submissions due to fears of misuse or limited benefit-sharing stemming from global economic or historical inequities. The subsequent development of the CARE principles for Indigenous Data Governance reflects these concerns (11). The INSDC is committed to providing mechanisms for appropriate recognition for those who are involved in data acquisition and submission as well as provenance of data. The INSDC sees the opportunity to expand its commitments to open science and responsible use of data through advances in the area of data attribution in its records. In the same way publication citations provide recognition for a variety of contributions to a research endeavor, a robust system of data attribution can do the same for those involved in sample acquisition, sequencing, processing, and submission of data. As the INSDC looks to the future, it is exploring opportunities to extend records in ways that support the intake and display of this information, as well as its role in a biodata economy that values data attribution.
Data citation and PIDs
Another element of open science and a biodata economy is data citation. In recent years, persistent identifiers (PIDs), also known as Digital Persistent Identifiers (DPIs), have emerged as a key resource for data citation. PIDs additionally support data and research integrity. From its beginning, the INSDC has provided persistent identifiers for all data in its databases in the form of accessions that are globally unique within their defined namespaces. Many mainstream publishers provide explicit mention of INSDC accessions when instructing manuscript authors in the citation of data used in their research (e.g. https://www.cell.com/biophysj/authors, https://www.biomedcentral.com/getpublished/editorial-policies#availability+of+data+and+materials, https://communities.springernature.com/posts/the-basics-of-data-citation). In recent years, Digital Object Identifiers (DOIs; https://www.doi.org/) have emerged as an alternative PID format used by various other database resources (e.g. Zenodo (https://zenodo.org/)). DOIs, which are managed by different registration agencies (e.g. DataCite (https://datacite.org/), CrossRef (https://www.crossref.org/)), provide persistent, machine resolvable, globally unique identifiers for a wide variety of data types. Users occasionally inquire of the INSDC whether it will provide DOIs for its database records or establish a resolving service for its accessions. At this time, the INSDC is monitoring evolving trends and requirements in the area of data citation and PIDs and it does not have plans to instantiate changes in 2025.
As the INSDC looks to the future, its commitment to global representation will not only shape its technical architecture and content but will also provide the collaboration with additional perspectives in addressing these topics of data federation, data attribution and data citation. Through its expanded membership, the INSDC will also be further prepared to support international efforts in public health surveillance and response. The INSDC retains its commitment to open science and data transparency and identifying new mechanisms to achieve its vision of being the premiere open access nucleotide sequence data sharing collaboration in the world, fostering access of nucleotide sequence data from all continents and regions, and providing broad recognition to those who generate and share sequence data.
Contributor Information
Ilene Karsch-Mizrachi, National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Building 38A, 8600 Rockville Pike, Bethesda, MD 20894, USA.
Masanori Arita, Bioinformation and DDBJ Center, National Institute of Genetics, Mishima, Shizuoka 411-8540, Japan.
Tony Burdett, European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Genome Campus, Hinxton, Cambridge CB10 1SD, UK.
Guy Cochrane, European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Genome Campus, Hinxton, Cambridge CB10 1SD, UK.
Yasukazu Nakamura, Bioinformation and DDBJ Center, National Institute of Genetics, Mishima, Shizuoka 411-8540, Japan.
Kim D Pruitt, National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Building 38A, 8600 Rockville Pike, Bethesda, MD 20894, USA.
Valerie A Schneider, National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Building 38A, 8600 Rockville Pike, Bethesda, MD 20894, USA.
Data availability
The International Nucleotide Sequence Database (INSDC) is freely accessible at https://www.insdc.org/.
Funding
The work of I.K.M., K.D.P. and V.A.S. was supported by the National Center for Biotechnology Information of the National Library of Medicine (NLM), National Institutes of Health; the work of M.A. and Y.N. was supported by the Research Organization of Information and Systems (ROIS) under the Ministry of Education, Culture, Sports, Science and Technology (MEXT) of Japan; the work of T.B. and G.C. was supported by European Molecular Biology Laboratory. Funding for open access charge: National Library of Medicine.
Conflict of interest statement. None declared.
References
- 1. Arita M., Karsch-Mizrachi I., Cochrane G.. The international nucleotide sequence database collaboration. Nucleic Acids Res. 2021; 49:D121–D124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Strasser B. Collecting Experiments Making Big Data Biology. 2019; University of Chicago Press. [Google Scholar]
- 3. Yuan D., Ahamed A., Burgin J., Cummins C., Devraj R., Gueye K., Gupta D., Gupta V., Haseeb M., Ihsan M.et al.. The European Nucleotide Archive in 2023. Nucleic Acids Res. 2024; 52:D92–D97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Sayers E.W., Cavanaugh M., Clark K., Pruitt K.D., Sherry S.T., Yankie L., Karsch-Mizrachi I.. GenBank 2024 update. Nucleic Acids Res. 2024; 52:D134–D137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Kodama Y., Ara T., Fukuda A., Tokimatsu T., Mashima J., Kosuge T., Tanizawa Y., Tanjo T., Ogasawara O., Fujisawa T.et al.. DDBJ update in 2024: the DDBJ Group Cloud service for sharing pre-publication data. Nucleic Acids Res. 2024; 10.1093/nar/gkae882. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Timme R.E., Karsch-Mizrachi I., Waheed Z., Arita M., MacCannell D., Maguire F., Petit Iii R., Page A.J., Mendes C.I., Nasar M.I.et al.. Putting everything in its place: using the INSDC compliant Pathogen Data Object Model to better structure genomic data submitted for public health applications. Microb. Genomics. 2023; 9:001145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Dugan V.G., Emrich S.J., Giraldo-Calderón G.I., Harb O.S., Newman R.M., Pickett B.E., Schriml L.M., Stockwell T.B., Stoeckert C.J. Jr., Sullivan D.Eet al.. Standardized metadata for human pathogen/vector genomic sequences. PLoS One. 2014; 9:e99979. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Yilmaz P., Kottmann R., Field D., Knight R., Cole J.R., Amaral-Zettler L., Gilbert J.A., Karsch-Mizrachi I., Johnston A., Cochrane G.et al.. Minimum information about a marker gene sequence (MIMARKS) and minimum information about any (x) sequence (MIxS) specifications. Nat. Biotechnol. 2011; 29:415–420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Griffiths E.J., Timme R.E., Mendes C.I., Page A.J., Alikhan N.F., Fornika D., Maguire F., Campos J., Park D., Olawoye I.B.et al.. Future-proofing and maximizing the utility of metadata: the PHA4GE SARS-CoV-2 contextual data specification package. GigaScience. 2022; 11:giac003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Wilkinson M.D., Dumontier M., Aalbersberg I.J., Appleton G., Axton M., Baak A., Blomberg N., Boiten J.W., da Silva Santos L.B., Bourne P.E.et al.. The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data. 2016; 3:160018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Carroll S.R., Garba I., Figueroa-Rodríguez O.L., Holbrook J., Lovett R., Materechera S., Parsons M., Raseroka K., Rodriguez-Lonebear D., Rowe R.et al.. The CARE principles for indigenous data governance. Data Sci. J. 2020; 19:43. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The International Nucleotide Sequence Database (INSDC) is freely accessible at https://www.insdc.org/.