Knowledge-Based Biomedical Data Science

Abstract

Knowledge-based biomedical data science involves the design and implementation of computer systems that act as if they knew about biomedicine. Such systems depend on formally represented knowledge in computer systems, often in the form of knowledge graphs. Here we survey recent progress in systems that use formally represented knowledge to address data science problems in both clinical and biological domains, as well as progress on approaches for creating knowledge graphs. Major themes include the relationships between knowledge graphs and machine learning, the use of natural language processing to construct knowledge graphs, and the expansion of novel knowledge-based approaches to clinical and biological domains.

INTRODUCTION

What Is Knowledge-Based Biomedical Data Science?

Knowledge-based biomedical data science (KBDS) involves the design and implementation of computer systems that act as if they knew about biomedicine.¹ There are many ways in which a system might act as if it knew something: For example, it might use existing knowledge to generate, rank, or evaluate hypotheses about a dataset, or it might answer a natural language question about a biomedical topic.

Knowledge-based systems have long been a theme in artificial intelligence research. Knowledge-based systems specify a knowledge representation—how a computer system represents knowledge internally—and one or more methods of inference or reasoning—how computations over those representations (perhaps combined with other inputs) are used to produce outputs. Classical descriptions of knowledge representation and reasoning systems [e.g., see Davis et al. (2)] characterize them by the ontological commitments a knowledge representation makes (i.e., what it can or cannot describe), which inferences are possible within it, and, sometimes, which of those inferences can be made efficiently. These issues remain useful in thinking about how knowledge representation and reasoning play a role in today’s data science environment.

This review provides some useful background knowledge on important KBDS concepts like ontologies, Semantic Web standards, and the distinction between knowledge bases and knowledge graphs (KGs). To provide context, we then describe some high-level applications of KBDS that were published prior to January 2020. Then, we describe our approach to reviewing the last year of KBDS research and present our findings. Finally, we conclude the review by discussing perceived barriers to and offering recommendations for conducting KBDS.

Ontologies

Ontologies are a vital component of knowledge representations. Knowledge representations are said to be grounded in a set of primitive terms, hereafter termed “primitives,” that specify those ontological commitments: the entities and processes that can be referred to by that knowledge representation. Computational ontologies are, then, collections of primitives relevant to a domain, often related to each other by explicit subsumption (subclass of) and meronomy (part of) statements.

Within the biomedical domain, ontologies [e.g., the Gene Ontology (GO) (3)] are community consensus views of the entities involved in biology, medicine, and biomedical research, analogous to how nomenclature committees systematize naming conventions. Knowledge bases created using primitives from community-curated ontologies, rather than idiosyncratic or single-use sets of primitives, provide significant advantages for reproducibility in scientific research, for interoperability, and for avoiding pitfalls in the modeling of knowledge. Primitives can be combined into assertions that express facts about the world. In the simplest case, an assertion links two primitives with a specific relationship. Consider, for example, that the Protein Ontology contains a primitive human p53 protein, the GO contains a primitive DNA strand renaturation, and the Relation Ontology contains a primitive participation that can link a physical entity to a process in which it participates. Those three primitives can be composed into an assertion that could be part of a knowledge base: Human p53 protein participates in DNA strand renaturation. Some KGs are grounded in terminological resources, such as UMLS (Unified Medical Language System), SNOMED CT (Systematized Nomenclature of Medicine, Clinical Terms), and the National Cancer Institute Thesaurus, that lack some aspects of a computational ontology.

Semantic Web Standards

While ontologies provide the primitive elements from which a knowledge representation is constructed, they are agnostic about the mechanisms by which entities are assembled into assertions. In 2011, the World Wide Web Consortium promulgated a collection of international standards for linking entities with shared meaning into assertions and managing collections of assertions, together termed the Semantic Web (https://www.w3.org/standards/semanticweb/). The Semantic Web builds on the standard Resource Description Framework (RDF) (https://www.w3.org/standards/techs/rdf), which provides a way to link three uniform resource identifiers (4) to specify a relationship between a pair of entities (forming an RDF triple). Collections of triples form a graph, where the entities are nodes and the relationships are edges connecting them. A computational mechanism for managing such collections is called a triplestore.

The Semantic Web standards also define RDF Schemas and a Web Ontology Language (OWL), which provide additional expressivity; SPARQL [SPARQL Protocol and RDF Query Language (5)], which provides a query language for interrogating RDF graphs or triplestores; and the Simple Knowledge Organization System, which provides a basic ontology. The OWL (6) is used to specify two important types of entities: instances and classes. Instances are particular entities or processes in the world (e.g., a particular molecule of p53) and classes are groupings of instances that meet a defined set of individually necessary and collectively sufficient criteria (e.g., human p53 proteins). As it lacks variables and quantification, OWL cannot express all logical statements about primitives; the subset of first-order logic that OWL can express is inspired by description logics (7).

Knowledge Base Versus Knowledge Graph

Collections of assertions, generally called knowledge bases, can be created, queried, and shared, and then in turn used by other systems that apply various inference methods to fulfill particular application needs. Knowledge bases that can be represented as graphs are often called “knowledge graphs.” While not all knowledge bases are implemented as graphs (e.g., some are databases where a table structure is used to make implicit assertions), in recent years, it has become very common to represent knowledge bases using Semantic Web standards, or at least to use knowledge bases that can produce and consume Semantic Web–compatible versions (8). For that reason, the terms “knowledge base” and “knowledge graph” are often used interchangeably. In 2012, Google announced its proprietary Knowledge Graph, which also popularized the use of the term (9). The literature sometimes contains terminological imprecision about the differences between knowledge bases, KGs, and ontologies; readers are referred to Reference 10 for a recent review and analysis of various published definitions.

In this review, we use the term “knowledge graph” (“KG”) and say that a KG is grounded in the set of primitives from which it is constructed. Some KGs also include a set of logical rules called axioms that relate assertions to each other (e.g., the human p53 protein is a subclass of the p53 protein class that is found in the organism human). Figure 1 shows a simple example of a biomedical KG.

Figure 1.

An example of a knowledge representation for building a biomedical knowledge graph. Boxes represent different types of data, which are drawn from ontologies and other sources of linked open data. Boxes are connected by directed edges and represent semantically and biologically meaningful relationships.

Biomedical Applications of Knowledge Graphs

KBDS does computation over KGs (and perhaps other inputs) to make inferences about biomedicine. While each of the works surveyed below addresses a different problem using a different technique, there are some common themes in the computational approaches to using KGs, including improving information retrieval, inferring new knowledge, and creating alternative representations of KGs. Each of these themes is discussed further below.

Information retrieval.

A major use of KGs is simply to organize knowledge for information retrieval. Such systems are designed to make it possible to find facts or evidence regarding a wide variety of topics, ranging in this review from cataloging traditional Chinese medical practices to decision support for pharmacovigilance. KGs have also been used to improve other forms of information retrieval, such as finding relevant publications in the literature.

Inferring new knowledge.

There are two primary ways that new information in KGs is automatically inferred: graph algorithms and logical reasoning.

Edge (or link) prediction is one class of graph algorithm that is widely used in KBDS. Edge prediction methods generally use the structure of a graph to identify edges that are likely but missing in the graph (11). In KGs, these are predictions of assertions about the world. This is a form of hypothesis generation and often includes an estimate of the confidence in the prediction. Many approaches to drug repurposing use edge prediction algorithms over KGs of drugs and diseases to identify new indications. Another broad class of graph algorithms does community finding, or identification of groups of entities in a KG that are similar or highly related to each other. For example, some approaches to disease subphenotyping apply community-finding approaches to KGs encoding information about patients.

The second way new information is inferred from KGs is through the use of logical reasoners. The Semantic Web OWL standard was designed to facilitate two important classes of reasoning over KGs: satisfiability and subsumption inference. Satisfiability inference checks to see if a class definition is logically satisfiable; it is possible for a KG to define a class that has no members (e.g., human p53 protein homologs in bacteria). Subsumption inference uses class definitions to identify all classes that are fully contained within some other class (e.g., all proteins are nitrogen-containing compounds). Specific reasoners, such as ELK (12) or HermiT (13), can be used to make these inferences with particular computational performance guarantees, which can be important in large KGs. Subsumption inference in particular is useful in KBDS because it makes explicit many edges that are otherwise implicit in KGs, and therefore it can improve the results of other algorithms that depend on the structure of the graph, such as link prediction or embeddings.

Alternative representations.

Machine learning, particularly in the form of artificial neural networks, is widely used in the KG context (14). One frequent application of neural networks to KGs is to create embeddings of entities or assertions by training autoencoder networks with inputs constructed from the KG. These embeddings can then be used to compute knowledge-based similarities between, e.g., drugs, proteins, and diseases. Neural network methods have also been used to identify parts of a KG relevant to question and answering (Q&A) for a given input question.

Known Challenges

Many challenges to designing, constructing, and utilizing KGs within the biomedical domain have been identified. Some of the more difficult challenges include (a) computational performance, (b) constructing KGs using expert curation or information extraction methods, and (c) meaningfully integrating disparate data sources. Each of these challenges is discussed further below.

Computational performance.

Computational performance is always a challenge when applying algorithmically complex methods to large volumes of data. Biomedical knowledge is very extensive, and broad biomedical KGs can contain billions of assertions. A wide variety of schemes have been proposed to address the computational complexity of both querying and inference over KGs (15).

Knowledge graph construction.

A few KGs, such as GO annotations (16), GO Causal Activity Models (GO-CAMs) (17), or Reactome (18), are constructed through intense expert curation efforts. However, several algorithmic approaches have been proposed to either augment these efforts or fully automate them. Automated approaches to KG construction fall into two broad classes: natural language processing (NLP) methods and data-driven construction. Data-driven KG construction can involve the integration of previously disparate resources or the direct analysis of large-scale datasets.

NLP methods propose to extract information from a set of documents to create KGs, for example, the Semantic MEDLINE Database (SemMedDB) (19). As NLP methods are all imperfect, these approaches are often focused on assessing the reliability of the information extracted, or on techniques to manage missing or erroneous assertions and other sources of noise.

Data integration.

Some data-driven approaches simply transform existing databases [e.g., DrugBank (20)] into KGs, which can be useful for tasks like facilitating adherence to FAIR (findable, accessible, interoperable, and reusable) research principles (21). More frequently, data-driven KG construction aims to integrate multiple sources of data into a single KG. If an integrated KG can ground the different sources in one set of primitives (ideally, from a community-curated ontology), then inference over the combined information can be facilitated. There are thousands of public, biomedically important databases (22); hence, integration approaches that support semantic compatibility are important and can lead to improved data quality, as incompatibilities sometimes signal errors (23).

METHODS OF REVIEW PROCESS

For this review, relevant literature was surveyed by searching PubMed and Google Scholar using the following phrases: “knowledge graph,” “biomedicine”; “knowledge graph,” “medicine”; “knowledge graph,” “medical”; and “knowledge graph,” “biology.” These terms were also searched by replacing “graph” with “base.” All paper types were eligible for inclusion (i.e., conference proceedings, dissertations, book chapters, peer-reviewed archived manuscripts, and published peer-reviewed manuscripts).

Results

The search phrases above returned 52 papers from PubMed and 7,752 papers from Google Scholar. Manual review of these papers was performed to identify those that were focused on the use or construction of KGs within the biomedical domain, which resulted in a reduced set of 174 papers. This set of papers was then further reduced to only include papers published or posted to public manuscript archives between January 2018 and December 2019 whose full-text version was publicly available at the time of writing, resulting in a final set of 83 papers. For additional details, please see Figure 2.

Figure 2.

Paper selection process outline. Combining the results from PubMed and Google Scholar queries, we narrowed down the list of papers using a two-step process. First, we performed a quick review to reduce the initial number of papers. Then, we closely inspected each paper, which helped us to arrive at the final set of 83 papers.

The final set of papers was further broken down by year of publication or presentation/submission, the manuscript type, and the journal or archive name. Among the 83 papers, 41 were published or posted online in 2018. The majority of 2018/2019 papers were published in conference proceedings (n = 38) or in peer-reviewed journals/books (n = 26), with the remaining papers posted as online preprints (n = 19). Among these, the majority of the 2018 and 2019 papers were published as conference proceedings (48.8% and 42.9%, respectively). The number of conference submissions decreased by 9.1% between 2018 and 2019; 2018 papers were primarily submitted to IEEE (Institute of Electrical and Electronics Engineers) (n = 10), whereas 2019 papers were submitted to ACM (Association for Computing Machinery) (n = 5), AAAI (Association for the Advancement of Artificial Intelligence) (n = 3), and IEEE (n = 3).

Organization and Presentation of Findings

The final set of papers fall into two broad categories, which are used to organize the remainder of this review: application of KGs (n = 53) versus construction of KGs (n = 30). This review then concludes by considering some nascent projects likely to be important in the near future, characterizing current barriers to building and using biomedical KGs, and making some recommendations. Information about each paper included in the final set is presented in Supplemental Tables 1 and 2, and broad themes spanning multiple papers are described below.

APPLICATIONS OF KNOWLEDGE GRAPHS IN BIOMEDICAL DATA SCIENCE

Applications of KGs are noted in a wide variety of biomedical domains, ranging from analysis of genomic data to clinical decision support. There is also a close relationship between KGs and biomedical NLP: KGs can be used to improve the quality of NLP, and NLP can be used to generate KGs from the literature. Thus, the application of KGs is further divided into three primary themes: (a) clinical, (b) biological, and (c) NLP. Supplemental Table 1 provides a high-level summary of the reviewed papers that used KGs to help solve a biomedical data science problem (References 24-79).

Clinical Applications

There were three primary themes identified within this domain in the papers surveyed, including the use of KGs to improve the retrieval of information from the literature or from large sources of clinical data, the use of KGs to provide confidence either by adding evidence to support phenomena observed in data or by completing missing information and deriving new hypotheses, and the use of KGs to improve the representation and presentation of complicated patient data or personal health information.

The first observed theme was the use of KGs to refine user queries and otherwise improve information retrieval from the literature or from an electronic health record (EHR) system. One study demonstrated that using KGs with traditional rule-based approaches for information retrieval performed better than using either KGs or rule-based approaches alone (24). Liu et al. (25) proposed a novel graph-based representation of patient data where entities were linked to concepts in a biomedical KG in order to enable querying based on domain knowledge. Other clinical applications of KGs in information retrieval included the finding that a KG-based component added to a larger system improved the ability of doctors to identify meaningful information from an EHR (26), a KG-based method for users to formulate queries within the context of relevant domain knowledge (27), and a system to rewrite user queries using domain knowledge (28).

The second observed theme was the use of KGs to address uncertainty by identifying relevant evidence. KGs have been leveraged to provide evidence for diagnostic assistance, clinical decision support machinery, or surveillance (29). For example, Bakal et al. (30) used SemMedDB and a subset of the UMLS to better predict treatments for and causes of different diseases, and Reumann et al. (31) found that using a KG was helpful for correctly identifying rare disease patients when examined using over 100 different queries. There were two papers that focused on surveillance. In the first paper, Bobed et al. (32) built a KG from an adverse drug reaction ontology and SNOMED CT as a means to improve pharmacovigilance. In the second paper, Kamdar et al. (33) built a KG using drug classes from the Anatomical Therapeutic Chemical Classification System and active ingredients in RxNorm to better understand opioid use patterns across the United States.

Also part of the second observed theme was the use of link prediction algorithms to discover missing knowledge or generate hypotheses (e.g., 25). To improve the identification of comorbid diseases, Biswas et al. (34) built a KG using the approach outlined by Alshahrani et al. (35) and then performed link prediction using an inductive inference method. Also using inductive inference methods, Callahan et al. (36) described a method for transforming OWL-encoded knowledge to create representations that were better suited to inductive inference tasks; the results were evaluated using queries against the Knowledge Base of Biomedicine (KaBOB) (37). Neil et al. (38) described a method for transforming KGs into graph convolutional neural networks and an attention model using independent learnable weights to measure each edge’s usefulness.

The final observed theme was the use of KGs to capture complex patient information for further processing. Xie and colleagues (39) created patient-specific traditional Chinese medicine KGs by mapping patient data to a general traditional Chinese medicine–specific KG. Shang et al. (40) described a method for creating visit-level representations of patients from EHR data and mapping to a drug–drug interaction KG to provide personalized medication combination recommendations. Three papers focused on how to improve the presentations of complex information or results. Huang et al. (41) developed a novel tool to enrich and visualize patient data by incorporating KG embeddings. Singh et al. (42) developed an interactive tool built on Cytoscape (43) to help users interact with their network data. And Queralt-Rosinach et al. (44) introduced a novel approach to create custom systematic literature reviews by formulating the review as a biomedical KG that contains information relevant to specific hypotheses provided by a user.

Biological Applications

In more basic research applications, broad themes included the use of KGs to produce embeddings for prediction or visualization in low-dimensional spaces (15, 45, 46), the use of link prediction methods over KGs to hypothesize previously unobserved relationships (36, 40, 45, 47-56), and the use of KGs to generate complex mechanistic accounts of experimental data. Several efforts combined these themes, particularly the use of edge embeddings to improve link prediction (34, 48, 53, 57, 58).

Node and edge embeddings provide a powerful method to suggest relationships among entities via similarity functions, in ways that complement path traversal through the graph. Many of the reviewed papers leveraged semantic similarity–inspired hypotheses to identify valuable drug–drug (40,47, 51), drug–target (51), or protein–protein interactions (46, 48), many of which were in turn applied to drug repurposing. Additionally, some of the papers converted KG-based embeddings into low-dimensional spaces in order to visualize clusters in two- or three-dimensional projections (41) to better display entities of interest.

In a particularly innovative approach, Tripodi et al. (45) combined gene expression time series and KG-based embeddings from a human-centric KG (59) to create specific and detailed hypotheses regarding mechanisms of toxicity. The resulting KG subgraphs that made up the hypothesized mechanisms were far richer than the black box toxicity predictions that would have been otherwise presented. Further, these subgraphs were also used to generate natural language narratives describing the mechanisms and their sources of evidence.

Natural Language Processing Applications

KGs have been used to improve NLP performance in a wide variety of genres, including summarization or information extraction from EHRs and Q&A systems (15, 26, 27, 29, 40, 60, 61). We observed that KG-derived embeddings used alone or in combination with other text-derived features (46) improved the performance of a variety of NLP tasks, including named entity recognition (62), coreference resolution (63), and relation extraction (64).

Additionally, several papers demonstrated the utility of KGs in information extraction methods. Ontologies can serve as formal dictionaries allowing for rapid indexing in named entity recognition and word sense disambiguation tasks (65, 66). Compared to lexicons, KGs offer far richer semantic context, identifying not only similar concepts but also rich collections of relationships that can be used to disambiguate or otherwise improve concept recognition in texts (65-68).

CONSTRUCTING KNOWLEDGE GRAPHS

Researchers have made substantial efforts in methods development and generated new results in the construction of KGs, as well as in extending, integrating, and evaluating them. Supplemental Table 2 provides a high-level summary of the papers surveyed on the construction of KGs (References 80-124).

Efforts to produce domain-specific KGs have been made in a variety of areas, including biodiversity (80-82) and the microbiome (https://ncats.nih.gov/translator), as well as for the purpose of enriching clinical data (83, 84). The papers surveyed on biodiversity focused specifically on how a KG could be created and linked to identifiers in the literature (80, 81) or other important biodiversity resources (82). In contrast, papers using KGs for clinical enrichment aimed to use them as a way to link clinical data to sources of evidence to provide support for clinical observations (84-86) or to help make the data more interpretable with respect to underlying biological mechanism(s) (83) for improved diagnosis (87).

Historically, NLP information extraction efforts have often been used to construct KGs; two novel methods to do so were published last year. One proposed a minimum supervision-based approach that combined traditional NLP pipelines for information extraction and biomedical context embeddings (88). The other focused on improving the extraction of biomedical facts from the literature by leveraging and refining specific seed patterns (80).

Although not a construction method per se, SemMedDB, one of the most widely used NLP-constructed KGs, was recently evaluated by Cong et al. (81). They found many contradictory assertions in a variety of fundamental relationship categories, underscoring the need to be cautious regarding noise in NLP-derived KGs. Finally, an ontology called BioKNO and a set of associated tools leveraging OWL were presented to assist scientists attempting to share data according to FAIR principles (82).

ORGANIZATIONAL EFFORTS IN KNOWLEDGE-BASED BIOMEDICAL DATA SCIENCE

Both US and European scientific institutions support KG efforts. Perhaps the most ambitious of these is the National Institutes of Health’s National Center for Advancing Translational Sciences’ Biomedical Data Translator project (https://ncats.nih.gov/translator). The goal of the Translator is a computational system that integrates sources of existing biomedical knowledge in order to translate clinical inquiries into relevant biomedical research results that synthesize elements of the integrated knowledge to directly answer the inquiry or generate testable hypotheses (83). A recent funding call targets $13.5 million per year for up to five years toward the construction of what they call “knowledge providers and autonomous relay agents” (89). Knowledge providers are systems that seek out, integrate, and provide high-value data sources within a specific scope of Translator-relevant knowledge, and presumably they would primarily use KGs to do so. Relay agents are to take clinical queries in a standardized format, dispatch subtasks to appropriate knowledge providers, receive responses back from knowledge sources (presumably also as subgraphs of a KG), and process responses using scoring metrics in order to return the most relevant and highest quality potential responses.

Investigators at the University of California San Francisco (UCSF), Google, the Lawrence Livermore National Laboratory, and the Institute for Systems Biology were recently awarded the National Science Foundation’s Convergence Accelerator Award (84). A total of 21 awards were given out, but the UCSF project was the only one that focused on solving biomedical and health-related problems. The awarded project, titled “A Multi-Scale Open Knowledge Network for Precision Medicine,” aims to integrate several sources of publicly available data in order to build what they term a “biomedical knowledge engine.” The long-term goal of this project is to create a resource that will help clinicians gain better insight into patient care, as well as provide a tool to aid biomedical research. The project will be developed using UCSF’s Scalable Precision Medicine Knowledge Engine (SPOKE) (85, 86). By incorporating additional data from the UCSF Information Commons (https://informationcommons.ucsf.edu/), SPOKE will extend Hetionet (87), which currently contains information from over 25 databases and links millions of drugs, diseases, and biological molecules. In collaboration with Google, SPOKE will eventually be made available to the public through Data Commons (90).

Elixir Europe (https://elixir-europe.org/) is a large multinational (and European Commission) project with the goal of managing and safeguarding the data generated by publicly funded life science research and integrating bioinformatics resources. In pursuit of those goals, Elixir’s interoperability platform promotes efforts in the European life science community to adopt standardized file formats, metadata, vocabulary, and identifiers, including work on the Semantic Web and the adoption of community-curated ontologies. The Elixir Core Data Resources (https://elixir-europe.org/services/tag/core-data-resources) are leaders in the production of interoperable knowledge resources and are widely used components of biomedical KGs.

Last year saw the announcement of GO-CAMs, a new approach to GO annotation (17). Although GO annotations are perhaps the most widely used knowledge representations in biomedical research, until GO-CAMs were introduced, the annotations could not be assembled into a coherent KG. While individual annotations implicitly linked GO classes to gene products, contextual information was lost: For example, the annotation process could not capture that cytochrome C participated in apoptosis only when it was in the cytoplasm. GO-CAM models and associated tooling are gradually replacing the traditional GO annotation process within the Alliance for Genomic Resources (81), meaning that future GO annotation will produce an increasingly rich, manually curated KG.

CONCLUSIONS AND RECOMMENDATIONS

As demonstrated by this review, the last year has seen tremendous amounts of new developments in both the construction and application of biomedical KGs. A significant number of the reviewed papers focused on the construction or application of KGs to solve problems within the clinical domain (e.g., providing evidence for traditional Chinese medicine, improving Q&A systems, and developing patient-level KGs). Another popular area observed in both biological and NLP-based applications of KGs was the development of novel methods to better utilize KGs (e.g., embedding algorithms to create alternative representations of data extracted from a KG and graph-based algorithms to improve information and relation extraction). Finally, we observed across all applications of KGs that while KGs provide an efficient way to present complex information (e.g., scientific and medical knowledge, biological interactions, and experimental results), user-friendly tools are still needed to help visualize and present this information. We also identified several challenges and barriers to construct and use KGs that emerged from these papers (and also from our shared research experiences) that have helped us to brainstorm solutions and recommendations. Each of these areas is further detailed below.

Barriers

Current barriers to constructing and using KGs include KG and data availability, data licensing issues (sometimes there are different licenses for each data source), a lack of agreed upon standards for constructing KGs, and dependency upon resources (i.e., software languages or applications) that may be obsolete, deprecated, or outside of users’ skill sets or areas of expertise.

The first barrier to using KGs is that building a KG is challenging, so the reuse of existing resources is highly desirable. Table 1 provides a list of currently available KGs. While each of these KGs is a valuable resource, each was developed to solve a specific problem and thus there may be challenges in applying it to new tasks. GO-CAMs have great promise, but currently only a relatively small number have been curated (17). Reactome (18) provides a very high quality and extensive KG grounded in community-curated ontologies, but it is limited in scope to biochemical reactions and pathways. KGs derived from manual or automatic data integration, such as KaBOB (37), PheKnowLator (59), Hetionet (87), SPOKE (85), Bio2RDF (91, 92), DisGeNET (93), BioGrakn (94), and the Data Commons Graph (90), all require different amounts of domain knowledge or technical expertise to utilize. KaBOB is grounded in community-curated ontologies, but licensing restrictions mean that users must download software and build the KG themselves, which requires expertise and computational resources. PheKnowLator is also grounded in community-curated ontologies, is deductively closed using ELK, and is publicly available, but it does not yet have a user interface. Bio2RDF, BioGrakn, the Data Commons Graph, DisGeNET, Hetionet, and SPOKE are constructed by combining several different types of data without a consistent set of primitives and are not fully grounded in ontologies. Finally, NLP-derived systems such as SemMedDB are noisy, making trustworthiness an issue. This review does not discuss all of the currently available biomedical KGs identified during our literature survey. Readers are referred to Table 1 for additional information.

Table 1.

Currently available biomedical knowledge graphs

Name (Reference)	Primitivesa	Domain	Backend	Last updated	Construction methodb
Bio2RDF (91)	Mixed concepts	Biomedical	Virtuoso	09/25/14	Mixed integration
BioGrakn (95)	Mixed concepts	Biomedical	Grakn.ai	09/27/19	Identifier-based integration
Data Commons Graph (90)	Concepts	Demographics, health, economics, crime, education, employment, housing	GO, Python	2019	Identifier-based integration
DisGeNET (94)	Ontology concepts (URIs)	Biomedical	RDF	01/01/19	Mixed integration
Hetionet (87)	Mixed concepts	Biomedical	Neo4j	07/08/19	Mixed integration
KaBOB (37)	Ontology concepts (URIs)	Biomedical	AllegroGraph	06/23/19	Semantic integration of OBOs
NGLY1 deficiency (44)	Mixed concepts	NGLY1 deficiency	Neo4j	08/08/19	Mixed integration
Ozymandias (101)	Ontology concepts (URIs)	Biodiversity	Blazegraph	2019	Linked data from ALA and ALD using CrossRef
PheKnowLator (59)	Mixed concepts	Biomedical	RDF, Python	09/25/19	Mixed integration
ROBOKOP (61)	Mixed concepts	Biomedical	GreenT, Neo4j	09/20/19	Mixed integration
SemMedDB (19)	UMLS concepts	Biomedical	Relational database	06/2018	Mixed integration; Semantic Knowledge Representation program and the UMLS
Sparklis (32)	Ontology concepts (URIs)	Pharmacovigilance	Neo4j, JSON API	01/2019	Semantic integration of OBOs
SPOKE (86)	Mixed concepts	Biomedical	Neo4j	2019	Mixed integration; extends Hetionet with other data in UCSF Information Commons

^aMixed concepts are constructed from ontology concepts and other nonontology concepts found in sources of biomedical data.

^bCreated using semantic integration (i.e., ontology-based) methods and methods that integrate data by matching up sets of identifiers (i.e., using other data sources like linked data, with standardized concept identifiers).

Abbreviations: ALA, Atlas of Living Australia; ALD, Australian Faunal Directory; API, application programming interface; GO, Gene Ontology; JSON, JavaScript Object Notation; NGLY1, N-glycanase 1; OBO, Open Biomedical and Biological Ontology; RDF, Resource Description Framework; UMLS, Unified Medical Language System; URI, uniform resource identifier.

KGs constructed from automatic methods also present significant barriers. KG construction from literature sources is usually framed as a relation extraction problem, where semantic triples are inferred from the text and then assembled into a KG. The correctness of this approach to KG construction can be determined either before the KG is constructed by evaluating the relation extraction process itself (a more traditional approach) or by evaluating the quality of the resulting KG itself. Evaluating the quality of the constructed KG allows for the use of the reasoning methods described above.

The final barrier is the lack of agreed upon standards for evaluating KGs. In fact, the lack of standards for constructing KGs within the biomedical domain may be one of the reasons why they are challenging to evaluate. Of the 25 reviewed papers on constructing KGs that evaluated their KGs, 4 provided qualitative evaluation (e.g., case studies, domain expert or focus group review of results, conceptual models, or prototypes), 5 provided quantitative evaluation (e.g., by applying machine learning to a specific holdout dataset or to a new dataset, by performing a KG completion task like edge prediction), and 16 provided both types of evaluation. One of the reviewed papers that provided both types of evaluation utilized crowdsourcing as a means to validate triples from their KG (95).

Recommendations

Based on the articles surveyed, we provide a brief list of recommendations below. Two of the most significant areas that deserve attention are the use of dense vector representations (embeddings) of concepts and the integration of KGs in NLP applications.

Knowledge graph embeddings.

Given the high volume of papers covering KG embeddings and NLP-based KGs, we felt it necessary to share recommendations gleaned from these works. A very active area of research is the use of KGs to create knowledge-based embeddings. Good embeddings are important to the performance of machine learning systems and therefore have wide applicability. KGs have been used to create embeddings for entities of many kinds (from genes to patients), as well as for relations, assertions, and more complex representations. Applications of these embeddings include prediction of drug–drug interactions, drug–target interactions, target discovery, and finding clinically relevant evidence. In addition to reusing embeddings from the surveyed papers, we recommend considering the tools described in the BioKEEN paper (96), which describes a Python-based library for training and tuning models to produce new knowledge-based embeddings.

Natural language processing-based knowledge graphs.

As previously described, a major theme of the reviewed literature is the use of text mining and NLP techniques to generate KGs. While this approach offers the potential for breadth missing from most manually curated KGs, it comes at the cost of a relatively large number of errors. Cong et al.’s (81) evaluation of SemMedDB, a widely used KG produced by the US National Library of Medicine, found nearly 500,000 inconsistent assertions, as well as a wide variety of apparently missing relationships. While they suggested methods that could be used to improve the quality of SemMedDB, our recommendation is to recognize that NLP-generated KGs are likely to be very noisy and need to be used with caution.

Future Work

The trends we observed in last year’s work are likely to continue. Applications of KGs will likely continue to involve generations of embeddings and other uses of KGs in machine learning. The close relationships between the development of NLP methods and KGs are likely to persist. The expansion of KGs to areas beyond molecular biology (e.g., biodiversity and traditional Chinese medicine this year) is also likely to continue. Some previous areas of research (e.g., KG-based enrichment analysis for gene sets) that did not see new results this year may also continue to be fruitful.

New methods applying KGs to analyze different sorts of experimental data such as images seem ripe for development. Robust and biologically meaningful ways to incorporate or add experimental data to biomedical KGs would help to improve the precision of predictions when used to generate novel hypotheses or as a means to interpret experimental results. Similarly, for clinical KGs, it will be important to find clinically meaningful ways to incorporate quantitative measures (e.g., laboratory test results and biomarker measurements) and outcomes from EHR data.

Other than Table 1 of this review, there is no central reference site or repository where one can access or identify a current list of available biomedical KGs. Researchers need a more systematic approach to the development, maintenance, and interoperability of biomedical KGs that facilitates sharing and the use of clear documentation. Existing efforts on general frameworks and tools like BioKEEN (96), Protégé (97), and BlazeGraph (98), in addition to open biomedical ontologies (99), are important and could be further extended toward standardized tool development. As KG evaluation remains challenging, new methods or benchmarks will be valuable.

A final area for future work is the development of tools for interacting with KGs. Protégé (97) and SPARQL are two of the most commonly utilized tools within the field, but both have limitations that make them unable to serve as comprehensive tool suites. While Protégé is useful for constructing and editing ontologies and for performing logical reasoning, it was not designed to efficiently handle very large KGs and has limited support for visualizing ontologies and KGs. SPARQL’s limitations in pathway search (100) and ease of use make its broader adoption challenging. Recently developed applications like the Data Commons Graph web browser (90), ROBOKOP (61), and Grakn Enterprise’s Knowledge Graph Management System and Workbase (https://grakn.ai/grakn-kgms) provide promising examples of well-crafted, sustainable user interfaces, but all of these applications are written in different programming languages and their proper use requires differing levels of programming expertise.