A FAIR Data Ecosystem for Science of Science

Abstract

This poster discusses Automated Research Workflows (ARWs) in the context of a FAIR data ecosystem for the science of science research. We offer a conceptual discussion from the point of view of information science and technology using several cases of “data problems” in the science of science research to illustrate the characteristics and expectations for designers and developers of a FAIR data ecosystem. Drawing from a 10-year data science project developing GenBank metadata workflows, we incorporate the ideas of ARWs into the FAIR data ecosystem discussion to set a broader context and increase generalizability. Researchers can use these as a guide for their data science projects to automate research workflows in the science of science domain and beyond.

INTRODUCTION

The fast growth of data and advances in computational tools have been loudly “revolutionizing” science (Atkins, 2011; Fortunato et al. 2018), but at the same time quietly changing the way research is conducted by expanding expectations of systems that service the needs for data findability, accessibility, interoperability, and reusability (FAIR) (Wilkinson et al, 2018). The term Automated Research Workflow (ARW) refers to the tools and techniques developed to support scientific investigations in meeting the demands for a FAIR data ecosystem and ensuring research reproducibility, replicability, and trustworthiness (NAS, 2022). ARWs “integrate computation, laboratory automation, and tools from AI in the performance of tasks that make up the research process such as designing experiments, observations, and simulations; collecting and analyzing data and learning from the results to inform further experiments, observations, and simulations” (NAS, 2022, p. 1). Although ARWs have accelerated scientific discoveries and yielded benefits to society (e.g., the rapid development of COVID-19 vaccines), the wider application and adoption of ARWs remains riddled with technical, social, cultural, educational, and policy-related challenges.

Science of science research has become increasingly computational, enabled in part by the explosion in trace data from the scientific enterprise and the sophistication and accessibility of AI/machine learning tools (Wang and Barabasi, 2021). However, before science of science data can be analyzed and run through computational workflows, the varied formats and structures of data from various publication and data repositories and the lack of linking between data requires a large amount of time and effort to acquire and process (Wilder-James, 2016; Qin et al., 2023). While the issues and questions raised in the NAS 2022 report are for all disciplines and at a more general level, this poster is going to discuss ARWs in the context of a FAIR data ecosystem for the science of science research.

The concept of a FAIR data ecosystem has been gaining attention from government funding agencies and research communities in the last decade (Wilkinson et al., 2016). The National Institutes of Health (NIH) strategic plan for data science defines a data ecosystem as “a distributed, adaptive, open system with properties of self-organization, scalability, and sustainability inspired by natural ecosystems” (NIH, 2018, p. 29). In this environment “data and resources become seamlessly integrated such that different data types and information about different organisms or diseases can be used easily together rather than existing in separate data ‘silos’ with only local utility” (NIH, 2018, p. 12). The National Science Foundation (NSF) has also strengthened the funding for building research data ecosystems in recent years (NSF, 2022). While funding agencies provide a wish list of essential characteristics of data ecosystems, many questions remain to be answered in specific disciplinary fields: what should a data ecosystem look like? What barriers exist for its implementation and use? What social and cultural impact may result from this transformed data environment for researchers? How do we (the information science and technology society) prepare workforces and reform educational programs for the ARW-driven data ecosystem? Clearly, different disciplinary fields have special sets of questions to address the ARW and data ecosystem challenge.

This poster offers a conceptual discussion from the point of view of information science and technology and in the science of science research as a case to illustrate the characteristics and expectations of a FAIR data ecosystem. We incorporate the ideas of ARWs into the FAIR data ecosystem discussion to set a broader context and increase generalizability. In this poster, we present the conceptual architecture of FAIR data ecosystems, introducing the concepts of research entities, artifacts, and a preliminary definition of data ecosystem in the context of science of science research. We then describe a proposed prototype of an ARW implemented for science of science in the biomedical context using GenBank metadata. We identify several pressing social, cultural, and educational challenges to implementing the vision of an automated research workflows to implement a FAIR data ecosystem for science of science research.

THE CONCEPTUAL ARCHITECTURE OF FAIR DATA ECOSYSTEMS

There are concepts important for the architecture of FAIR data ecosystems:

Research entities:

refers to authors and their affiliations, publications, datasets, patents, and grant awards – the primary objects and interests of science of science research. Research entities are often relatively stable and required to be consistent and identified globally uniquely. The metadata for these research entities are recorded as structured data and identified by a standard or local identifier. For example, authors can be globally uniquely identified by the Open Researcher and Contributor Identifier (ORCID), publications and datasets by Digital Object Identifier (DOI) and/or PMID (used for publications in PubMed), and NIH grants by project number. The entity representation promotes linking by relations, such as the OpenAlex entity graph (Priem et al., 2022).

Artifacts:

refers to the input, computational codes, workflows, models, pipelines, and output that are used in/generated from a research lifecycle. Unlike research entities, artifacts are much more dynamic as they are constantly being revised/tuned to achieve the optimal performance or outcome. Metadata about these artifacts including version control is critical for provenance, verification, and reusability, which are significant properties of trustworthy computational analysis (Wing, 2021).

Data ecosystem:

refers to an architecture for acquiring, organizing, linking, and sharing research entities and associated artifacts in databases by using automatic methods. The data ecosystem is built on the cloud/virtual infrastructure in which research entities and artifacts are represented by heterogeneous graphs for data discovery, selection, extraction, and other operations for researchers to obtain analysis-friendly datasets. The entity and artifact graphs are developed through an ontology that represents the knowledge networks of collaborations, communities, and innovations.

A FAIR DATA ECOSYSTEM PROTOTYPE USING GENBANK

The GenBank metadata analytics project (Bratt et al., 2017, Qin et al., 2022) has accumulated diverse trace data sources for molecular sequence submissions and associated publications, grants, and patents. Using this data collection, we aim to seamlessly integrate and link metadata for research entities and artifacts to create graphs not only for more effective discovery and use of data and knowledge, but also for tracking data and workflows used in research to ensure reproducibility and transparency.

The prototype will follow the FAIR principles and emphasize integration and linking of key research entities for use with metadata analytics. As the data sources already exist in distributed data systems, e.g., authors in ORCID, publications in PubMed and Microsoft’s Academic Graph, this prototype will focus on linking mechanisms for connecting research entities and resources in the data and knowledge spaces of the data ecosystem. As a data ecosystem, algorithms and tools will be important for optimizing the automation of data ingestion, processing, transformation, and representation, so that the data in this system stay up to date. A requirement for a FAIR data ecosystem is that not only must the data be FAIR but also the artifacts generated from data design and creation process be FAIR. Standardized notes, annotations, workflows, code files would need to be integrated for data and code provenance purposes. Ontology models will represent the conceptual architecture of this prototype for research entities, artifacts, as well as their relations. This area of work will be guided by three core principles: express knowledge in a sufficiently precise notation, the knowledge representation scheme meets the criteria of adequacy and expressiveness, and reasoning and problem solving are based on the facts represented by the schemes (Qin, 2020). The prototype data ecosystem will include not only baseline functions of findability, accessibility, interoperability, and reusability for data and artifacts but also advanced functions such as knowledge representation using “a formal, accessible, shared, and broadly applicable language” (Wilkinson et al., 2016, p. 4).

CONCLUSION

A data ecosystem is a new way to view the vast digital data and has increasingly become a new wave in the organization, management, and services for digital data. There have been many lessons learned from the digital library initiative almost 30 years ago. In this round of revolutionary changes, some lessons from the digital library initiative are still valid, e.g., community building and outreach, but many more are new and different from 30 years ago as we discussed earlier in this text. We hope this poster will stir up discussion and rethinking on the implications of data ecosystems among the ASIST community members and beyond.