Multiscore, a gene ranker powered by artificial intelligence and real-world clinical data, shows high sensitivity for the molecular diagnosis of Mendelian disorders in nearly 10,000 exomes and genomes

Abstract

A challenge for clinical exome and genome sequencing (ES/GS) analysis is correlating the clinical presentation of the cases being tested with known gene-phenotype associations (GPAs). We developed Multiscore, a gene prioritization tool, to facilitate gene-level predictions of phenotypic fit. Multiscore combines data inputs and algorithms to generate similarity subscores that feed a random forest (RF) classifier trained to predict the probability of association between the patient’s clinical features and the gene. The reference GPAs are extracted from: (1) OMIM, (2) patient descriptions in the literature, and (3) GeneDx (GDx) clinical data. We used 9,989 ES/GS cases to assess performance of the tool in combination with genotype filtering. Genotype filters rendered an average of 173 genes with variants requiring clinical review. Multiscore prioritized the reported positive gene with a median rank of 3 and mean rank of 6.35. The average recall (sensitivity) of Multiscore was 33% in the top 1, 69% in the top 5, 83% in the top 10, and 93% in the top 20 ranked genes. Multiscore was able to handle non-exact HPO term matches allowing the use of real-world clinical data. 74 genes lacking OMIM entries were prioritized using only the GDx and literature datasets. Multiscore allows the phenotype review to prioritize the most relevant genes, increasing case throughput and broadening access to diagnoses for patients.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00439-025-02797-z.

Introduction

One of the foremost challenges of clinical exome and genome sequencing (ES/GS) is efficiently filtering down the thousands of variants identified in a patient’s DNA to a relevant few that could be contributing to an individual’s clinical symptoms (i.e., their phenotype). This is usually accomplished by combining analysis of variant pathogenicity informed by multiple computational algorithms (Choi and Chan 2015; Jaganathan et al. 2019), frequency in the general population (Karczewski et al. 2021), and clinical review of the phenotype, which in turn requires a deep understanding of gene-disease relations. Clinical information can be translated into a standard vocabulary, the Human Phenotype Ontology (HPO) (Robinson et al. 2008). Assessment of multiple databases of gene-phenotype associations (GPAs) such as the Online Mendelian Inheritance in Man (OMIM) (Amberger et al. 2009), Orphanet (Aymé 2003) and GeneReviews (https://www.ncbi.nlm.nih.gov/books/NBK1116/), alone or in combination with literature reviews, may be needed. This clinical review remains a difficult task, complicated by long GPA sets, identification of relevant phenotypes in the literature and patient records, questions around non-exact phenotype matching, and consideration of disease- versus gene-level information.

Automated tools have been developed to facilitate clinical variant review and assess phenotypic overlap. However, tools that rely on case phenotypes or gene phenotypes alone (Jagadeesh et al. 2019; Köhler et al. 2009; Robinson et al. 2020) are limited by information on those published curated databases (Amberger et al. 2009; Ayme 2003), with only a few recent attempts at using electronic health records to derive phenotype data (De Freitas et al. 2021; Yang et al. 2023). Even when using real patient data, cohorts are typically part of gene discovery programs or cases from the literature, which include deep phenotyping by genetic specialists and do not always translate to the clinical information provided to clinical genomics testing laboratories performing case analysis (Li et al. 2019; Yang et al. 2023). Many more tools combine the phenotypic fit assessment with variant pathogenicity predictors (Birgmeier et al. 2020; Javed et al. 2014; Li et al. 2019; Mao et al. 2024; Zemojtel et al. 2014) or protein, gene, or pathway interactions (Alsentzer et al. 2025; Cornish et al. 2018; Rao et al. 2018; Smedley et al. 2015; Yang et al. 2015; Zhao et al. 2020). Newer tools like AMELIE (Birgmeier et al. 2020) incorporate a literature search to construct associations between gene, variants, and phenotypes, thus addressing a major limitation of the previous models at the potential expense of deprioritizing unpublished variants (Yuan et al. 2022).

Ultimately, reliance on public datasets, small study cohorts, and published literature may limit the identification of variants of possible clinical interest, as the data sources are skewed and do not necessarily represent the diversity of phenotypic manifestations reported in a clinical testing population. It is important then to leverage real-world data from a large and diverse clinical testing population using an approach that facilitates efficient abstraction of large amounts of clinical information and quantitative integration of these data into case analysis. To this end, we developed Multiscore, a gene prioritization tool informed by a large dataset of well-characterized individuals in addition to external data sources, combined with an ensemble of multiple similarity scores complementing each other using machine learning (ML). This tool highlights phenotypes that could be missed by direct comparison with published curated datasets and allows the phenotype review process to focus on the variants in the most relevant genes, increasing the likelihood of timely diagnoses without impacting phenotype expansions and discovery of new disease associations.

Methods

Multiscore tool

We chose a random forest (RF) model to learn the relationship between multiple similarity scores derived from the clinical presentation of a patient and the positive/not positive association between that patient and a gene. The RF model is a foundational and relatively simple ML solution to learning from numerical tabular data, and it has been shown to be a good approach to evaluate gene-disease associations (Huynh-Thu et al. 2010; Chen and Ishwaran 2012; Acharjee et al. 2020; Hu and Szymczak 2024; Karunakaran et al. 2025). In our approach, clinical information from the literature and patient records are translated into HPO terminology and phenotype vectors using a suite of internally developed and trained tools. Then, the patient’s information is compared to the GPA knowledgebase using multiple similarity scores generated by a Scoring Engine. These subscores feed an RF model which predicts the probability of a phenotypic match with a list of genes (Fig. 1). The range of possible subscore values is [0.0,1.0] with 0.0 representing no match and 1.0 representing a perfect match. The genes associated with the patient variants passing genotype filters are scored. The probability scores from the RF model are sorted to yield a prioritized list of genes. The rank is the fractional rank of the gene against all tested genes. The range for this value is (0.0, 1.0] where higher numbers indicate a higher rank. Genes with equal probability scores get the highest possible ranks. In this report, we simplify the results using a rank of 1,2,3,… to indicate the gene with highest, second-highest, third-highest, etc. RF model probability.

Fig. 1.

Overview of the genotype and phenotype analysis schema and details of Multiscore. Genotype and phenotype analyses are handled independently. Patient genotype is used for filtering (Retterer et al. 2016), and phenotype informs the final selection of variants for reporting during clinical analysis. Multiscore is shown in the blue dotted box. Patient clinical information is fed to the Scoring Engine as structured HPO terms and unstructured medical text. All genes with variants that pass the genotype filters are analyzed in the Scoring Engine and the RF Multiscore model. The Scoring Engine leverages three datasets of GPAs (top left, green circles): the GDx dataset, Lit dataset, and HPOA dataset. The Scoring Engine also leverages four scoring algorithms (top right, purple circles): Jaccard similarity, Hybrid Relative Semantic Similarity (HRSS) (Wu et al. 2013), word2vec (Mikolov et al. 2013) cosine similarity, and doc2vec (Le and Mikolov 2014) cosine similarity. The GPA datasets and scoring algorithms are combined in pairs to create eight subscores (represented by gray lines between the pairs): Jaccard GDx, HRSS GDx, word2vec GDx, Jaccard Lit, HRSS Lit, word2vec Lit, HRSS HPOA, and doc2vec* (Table S1). For each gene passed to Multiscore, the Scoring Engine computes these eight subscores, and these subscores are subsequently passed as features to the RF model. The RF model computes the probability of a positive association between each gene and the patient case. The probabilities for each gene are sorted to create a ranked list of prioritized genes. *doc2vec GPAs are vector embeddings of individual sentences that contain phenotypic information from gene-linked publications in the Literature Surveyor corpus (Supplementary Methods). The doc2vec neural net that embeds the text was also trained from this corpus by Literature Surveyor. The structure of these doc2vec GPAs is distinct from the (gene, HPO code) pairs that comprise the GPAs used by Jaccard, HRSS, and word2vec, hence, we use a dash-dot-dot line to link the Lit dataset and the doc2vec scoring algorithm.

Data dependencies and component algorithms

Multiscore gene-phenotype associations (GPA)

We used three sources of data: (1) disease-phenotype annotations extracted from OMIM (purl.obolibrary.org/obo/hp/hpoa/phenotype.hpoa) (HPOA dataset), (2) patient descriptions obtained from the relevant published literature (Lit dataset), and (3) a structured clinical dataset describing probands with a positive finding identified by exome or genome sequencing at GeneDx (GDx dataset). GPAs are stored as (gene ID, HPO term) pairs for all three datasets. doc2vec text embedding GPAs were also created by Literature Surveyor (see below) and stored as (gene ID, sentence embedding) pairs for the Lit dataset.

Clinical information for the GDx dataset was obtained from clinical notes submitted with the sample, ICD-10 codes, and relevant client communications. A positive finding was defined as the presence of variant(s) matching the accepted mode of inheritance in a disease gene with good phenotypic fit and classified as pathogenic or likely pathogenic according to American College of Medical Genetics classification criteria (Richards et al. 2015). It is uncommon to receive feedback from clinical teams, but when we do, we incorporate that information in our database such that the finding may not be considered diagnostic if it is not a good phenotypic fit. Cases with dual molecular positive findings and cases where the positive finding was a multi-gene copy number variant were excluded from the GDx dataset, model training, and model testing.

GeneDx knowledge reference

GeneDx harbors a dynamic knowledge base of phenotype information, variant classifications, and gene-disease associations that inform our reporting strategy. Trained curators review gene-disease associations reported in the literature and those already curated in OMIM and classify them as either candidate (equivalent to Clingen’s limited evidence for gene-disease validity) or validated (equivalent to Clingen’s moderate or higher association strength). The outcome of the assessment, mutation spectrum, functional domains, and phenotypes are stored in a database. Some disease entries in the GeneDx knowledge reference represent more than one disease in OMIM (Amberger et al. 2009). Diseases were consolidated or “lumped” following the guidance suggested by Thaxton et al. (2022). Disorders that shared a common inheritance pattern, mechanism of disease, and phenotypic features within and/or between families segregating the same variant(s), are reviewed by curators and lumped. These lumped disease entries represent a continuous spectrum of diseases, and based on case information, it is not always possible to say where in the spectrum a specific patient may fall. For example, the CEP290 gene is associated with five diseases in OMIM: Bardet-Biedl syndrome (MIM 615991), Joubert syndrome (MIM 610188), Leber congenital amaurosis (MIM 611755), Meckel syndrome (MIM 611134), and Senior-Loken syndrome (MIM 610189). These diseases share overlapping clinical features, inheritance, and mechanism of disease and thus were lumped under the CEP290-related ciliopathy umbrella. Phenotypes associated with individual OMIM entries are aggregated when they are lumped. When lumping is not possible after curator review, the entries are kept separate.

Literature Surveyor

There are over 100,000 new potentially relevant publications in PubMed every month. Literature Surveyor was developed at GeneDx to leverage natural language processing methodologies and automate the aggregation, structuring, and modelling of biomedical literature for gene-phenotype discovery. Literature Surveyor performs three main functions: (1) downloading and processing all articles in the corpus, (2) calculating gene-phenotype annotations found in the corpus, and (3) training a doc2vec model to learn and generate embeddings from gene and phenotype-rich articles within the corpus (Supplementary methods). The sources of documents comprising the corpus include MEDLINE/Pubmed (https://ftp.ncbi.nlm.nih.gov/pubmed/updatefiles/), PMC Articles from NCBI (https://ftp.ncbi.nlm.nih.gov/pub/pmc/), and GeneReviews (https://ftp.ncbi.nlm.nih.gov/pub/litarch/ca/84/). Documents undergo HPO term identification via txt2hpo (see below and https://github.com/GeneDx/txt2hpo) and HPO terms are linked to genes.

word2vec model

word2vec (Mikolov et al. 2013) is a context-based natural language processing method that measures a quantitative similarity between word pairs using word embeddings - the process of converting a word into a vector of numbers (“word to vector”). The vector representation of the word can then be compared to other vector-embedded words using standard algorithms like cosine similarity. For the word2vec model to learn the context of a word (surrounding words that typically appear near a term in a document), it trains a neural network on an existing corpus of text which consisted of phenotype terms in the HPO. Because related words tend to have similar contexts, their vector embeddings tend to cluster together in feature space, and this distance between words can be measured using cosine similarity. In Multiscore, the word2vec similarity is implemented in phenopy using the gensim library (https://github.com/RaRe-Technologies/gensim). For a given dataset, the word2vec subscore is the similarity between the list of patient phenotypes and the list of dataset GPAs for a single gene, computed as the cosine similarity between the average vector embeddings for each of the two lists using genism.models.KeyedVectors.n_similarity.

doc2vec model

doc2vec (Le and Mikolov 2014) is a neural network-based natural language processing algorithm for semantic similarity measurements between documents. As the name implies, doc2vec creates document embeddings, where an entire document is converted into one vector (“document to vector”). The doc2vec model is trained in-house to learn the embeddings of medical text describing patients diagnosed with genetic disease using the gensim library (Supplementary Methods). In our implementation of the Literature Surveyor doc2vec model in Multiscore, a document is defined in two ways: (1) a single sentence from gene-linked publications in the Literature Surveyor corpus, or (2) the medical text of a single patient case. For a given gene, the doc2vec subscore is the maximum cosine similarity value between all gene-linked sentences and the patient medical text.

txt2hpo

txt2hpo is an internally built tool used to identify HPO terms within text (https://github.com/GeneDx/txt2hpo). Text input is split into phrases by identifying punctuation using scispaCy (https://allenai.github.io/scispacy). Spellchecking is performed based on the Norvig method (https://norvig.com/spell-correct.html). The vocabulary is based on biomedical language and common English words from the spaCY “en_core_sci_sm” library and is supplemented by the terms found in the HPO. Next, tokenization, stemming, and normalization is performed: the text is divided into individual words known as tokens, the tokens are reduced to their base or root form (e.g., plural form to singular), and stop words and punctuation are removed. HPO terms are identified from this preprocessed text using a search tree built from HPO terms and synonyms. Negated terms indicate the absence of the condition and is implemented by negspaCY (https://github.com/jenojp/negspacy). For ambiguous terms, e.g. “ASD”, which can refer to “autism spectrum disorder” or “atrial septal defect”, the HPO term with the highest context match will be chosen. Context matching is determined by cosine similarity between embeddings of the nearby input text (token width = 8) and the text definition of the HPO terms. The embeddings were calculated by a doc2vec model.

Data processing

All three data sources (OMIM, Literature, and GeneDx Knowledge Reference) were processed to produce GPAs which comprise all phenotypes associated with each gene. GDx dataset-GPAs were obtained from the list of unique phenotypes across all the patients with single gene positive findings in that gene in the dataset. Literature dataset GPAs were collected from phenotypes extracted from publications using the gene2pubmed database. HPOA dataset GPAs were collected from the disease-level OMIM phenotype annotations (https://github.com/obophenotype/human-phenotype-ontology/releases/tag/v2023-09-01/phenotype.hpoa). Prioritization scoring also depended on the word2vec (Mikolov et al. 2013) model and doc2vec (Le and Mikolov 2014) model trained in-house (Supplementary Methods).

Model design

Multiscore considered the phenotypic profile of a patient against a list of genes. The clinical information of a patient and genotypically relevant genes were passed to the Scoring Engine. The Scoring Engine is responsible for computing multiple similarity scores between the clinical description and the relevant genes. For each gene, data inputs and algorithms were mixed in combination to generate eight patient-gene similarity subscores (Fig. 1, Table S1) using (1) the phenopy (https://github.com/genedx/phenopy) python class phenopy.score.Scorer for scoring HRSS, Jaccard, and word2vec similarities plus (2) code developed in Multiscore for scoring doc2vec similarity. The eight subscores were then fed to the RF classifier (Breiman 2001), which yielded a probability of association between a patient’s clinical description and a gene. The genes were then ranked by this probability score.

Multiscore allows non-exact HPO term matching by using: (1) doc2vec to convert patient medical text to vector embeddings and compare the embedded text to sentences from publications in the Literature Surveyor corpus, (2) Hybrid Relative Semantic Similarity (HRSS) to assess the distance between HPO terms in the HPO graph and their information content (Wu et al. 2013), and (3) word2vec (Mikolov et al. 2013) to interpret the local context of HPO terms in literature. Exact HPO term matching between the case and the gene knowledge dataset is evaluated using Jaccard similarity. Additional information regarding the implementation of these tools in Multiscore can be found in the Supplementary Methods.

The classifier did not learn information related to phenotypes or Mendelian disease; rather, it learned the relationship between the combinations of feature subscores that paired with a positive finding in a patient case. By training a classifier model on this combination of subscores from various datasets and algorithms, Multiscore learned to identify a strong signal of association between patient and gene from an ensemble of weaker signals.

We evaluated the impact of incorporating the GDx dataset by looking at the number of individuals included to create the gene reference information and the term frequency-inverse document frequency (TF-iDF) of all the HPO terms in each gene group. TF-iDF measures term importance within the cases associated with one gene, combining the frequency of a term within those cases with the presence of that term within all genes in the GPA corpus.

Model training and testing

Training and testing were devised as a time-series experiment: the annotation data (GDx, HPOA, and Lit) and the subscores for training and testing cases were collected in monthly batches (Figure S1). The testing case dataset was separate and distinct from the training case dataset. The training dataset consisted of 3,654 positive cases analyzed at GeneDx between 11/1/2021-7/31/2022 and the testing dataset consisted of 9,989 positive cases analyzed between 8/1/2022-1/31/2024 (Table 1). 327 cases with dual diagnosis (83 in testing set, 224 in testing set), and 2,700 cases where the diagnostic finding was a multi-gene copy number variant (399 training, 2,301 testing) were excluded from these experiments. The RF model was trained using the Python package scikit-learn version 1.6.1 using the module sklearn.ensemble.RandomForestClassifier. Additional details regarding model training are described in the Supplementary Methods and Table S2.

Table 1.

Characteristics of the GDx dataset used for training and testing multiscore

	Multiscore training data	Multiscore testing data
Number of cases	3,654	9,989
Age at testing (mean in years, SD)	9.66, 11.3	9.62, 11.5
Sex assigned at birth
Male (%)	53.7%	54.4%
Female (%)	46.3%	45.5%
Unknown (%)	0.0%	0.04%
Genetic ancestry prediction*
EUR (%)	51.60%	49.10%
AMR (%)	24.60%	25.80%
AFR (%)	11.10%	12.50%
MDE (%)	6.94%	6.87%
SAS (%)	3.07%	3.13%
EAS (%)	2.69%	2.54%
Represented phenotypes+
Arrhythmia	7.4%	7.8%
Autism	23.5%	26.7%
Cardiomyopathy	1.5%	2.0%
Cerebral Palsy	6.1%	5.6%
Connective tissue abnormalities	13.8%	14.8%
Epilepsy	26.7%	26.6%
Hearing loss	10.9%	11.0%
Multiple congenital anomalies	80.7%	78.1%
Neurodevelopmental abnormalities	75.5%	74.6%
Neuromuscular abnormalities	61.5%	57.2%
Skeletal dysplasia	1.6%	1.6%

Phenotype category definitions are described in Table S3. These categories are not exclusive to each other and are not meant to encompass all the terms in the HPO tree. Note that this cohort only includes cases with a positive finding and the represented phenotypes should not be confused with diagnostic yield.

EUR European, AMR American admixed, AFR African, MDE Middle Eastern, SAS Southeast Asian, EAS East Asian.

*Genetic ancestry prediction was performed using principal component analysis of the sample against a reference sample set from 1000 Genomes Project plus middle Eastern patients from GeneDx. For details, see Table S3 in Lake et al. 2017.

⁺Represented phenotypes show the percentage of patients in each data set where the patient has one or more phenotypes within that category.

All genes with variants passing the genotype filters in a case were scored. The genes were ranked by the Multiscore probability and by the values of the eight subscores to compare the performance of Multiscore to each of the independent subscores.

In the training set, the original ratio of negative genes to positive genes was 154:1. The negative genes in a case are those with variants passing genotype filters but not reported as positive. The negative genes in the training set were downsampled to a negative to positive ratio of 5:1 to optimize the performance of the model (Supplementary Methods).

We used “recall at k” (or “sensitivity at k”) to measure how well Multiscore prioritized the positive finding gene identified by standard clinical analysis. The recall at k for a case describes whether the positive finding was ranked in the top k genes tested. We took the average over groups of cases to give the average recall at k. In this study, where we tested cases with one relevant gene per case, average recall at k is equivalent to the fraction of patient cases where the reported positive finding gene ranked in the top k genes by Multiscore.

Benchmarking

We analyzed the same 9,989 cases using two other phenotype-only prioritization tools: Phrank (Jagadeesh et al. 2019) using the GDx GPA knowledgebase, Phrank (Jagadeesh et al. 2019) using the HPOA GPA knowledgebase, and LIRICAL (Robinson et al. 2020). Phrank (Jagadeesh et al. 2019) was implemented in GORPipe (Guðbjartsson et al. 2016; Supplementary Methods). For Phrank GDx, the GDx knowledgebase was used to test cases in monthly batches as in Multiscore. For Phrank HPOA (Jagadeesh et al. 2019) and LIRICAL (Robinson et al. 2020), the HPOA release from 9/1/2023 was used for scoring all batches. For LIRICAL (Robinson et al. 2020), ORPHANET (Aymé 2003) GPAs from the 9/1/2023 release were included to supplement the OMIM entries. LIRICAL v2.0.2 was used to score cases using yaml files built for each case. LIRICAL (Robinson et al. 2020) was not run on 4 fetal cases with undetermined sex in the clinical information.

Multiscore implementation in a clinical setting

Genotype analysis filters (Retterer et al.2016) focused on variant level indicators of pathogenicity and included: (1) segregation using joint calling with the parental genotypes, when available (i.e., de novo, compound heterozygous status), (2) variants reported in HGMD (Stenson et al. 2020; GeneDx proprietary version), ClinVar (Landrum et al. 2014), or in the GeneDx knowledgebase, and (3) rare unclassified variants predicted to impact protein function (McLaren et al. 2016). Variants with allele frequency > 0.01 in gnomAD (Karczewski et al. 2020) are excluded. Gene-disease associations are determined by the GeneDx knowledge reference. GeneDx’ approach to ES/GS analysis keeps genotype and phenotype analysis separate to allow identification of disease associations not yet described in the literature and avoid penalization of newer associations. Genotype-based filters create a pool of variants for clinical review. All genes with variants passing genotype filters are scored using Multiscore. Multiscore was not used to filter variants; instead, gene ranks of the Multiscore RF probabilities were displayed as an additional tool to inform that review. Since Multiscore assessed the phenotypic fit of genes, all variants in a gene are assigned the same score (i.e., compound heterozygous).

Results

Multiscore consistently prioritized genes with positive findings in 9,989 retrospectively analyzed cases

The average number of genes with variants requiring review in this study was 173 per case (90.1 for trios and 307 for non-trios) (Table 2). Multiscore prioritized the reported positive gene with a median rank of 3 and mean rank of 6.35 (standard deviation 10.2). Multiscore prioritized the positive finding in the top 1 gene in 33.1% of cases, in the top 3 genes in 57.2% of cases, in the top 5 genes in 69.1% of cases, in the top 10 genes in 83.5% of the cases, and in the top 20 genes in 93.3% of the 9,989 cases in the testing dataset. The recall performance was lower for non-trio cases which had less informed genotype filtering due to incomplete segregation information, but Multiscore still prioritized the positive finding in the top 10 genes in 76.4% of cases. The average number of genes highlighted by Multiscore as a possible phenotypic match was 79.0 (35.7 in trios, 150 in non-trios).

Table 2.

Multiscore performance for all cases, trio cases, and non-trio cases

	All cases	Trios	Non-trios
Cases (n)	9,989	6,190	3,799
Average recall at k
1	33.1%	36.3%	27.8%
2	47.9%	52.8%	40.0%
3	57.2%	62.6%	48.3%
5	69.1%	74.5%	60.4%
10	83.5%	87.9%	76.4%
15	89.8%	93.2%	84.3%
20	93.3%	96.2%	88.7%
Median rank	3	2	4
Mean rank	6.35	4.99	8.57
Rank standard deviation	10.2	7.35	13.42
Mean number of genes per case	173	90.1	307
Median number of genes per case	108	85	284
Mean number of genes prioritized by Multiscore	79.0	35.7	150
Median number of genes prioritized by Multiscore	44	33	138

Average recall at k, median rank, mean rank, and rank standard deviation all refer to prioritization of the positive reported gene

We compared the performance of Multiscore to rank the positive gene against each individual subscore for k = 1 through k = 50 (Table S4, Figure S2). The average recall at k was always highest for Multiscore, followed by word2vec GDx, then by HRSS HPOA, until k = 34, where the recall for HRSS GDx surpasses HRSS HPOA.

The overall wins tally of Multiscore and each subscore is shown in the top bar chart of Fig. 2. Multiscore gave the highest ranking for the positive finding compared to the subscores in 1,520 cases and tied for highest in an additional 3,026 cases, for a total of 4,546/9,989 (45.9%) cases tested, followed by word2vec GDx, which was the highest-ranking score in 1,394 cases and tied for highest in 2,229 cases, for a total of 3,623/9,989 (36.6%) cases. For cases where a score was not the highest rank (or tied for highest), we tracked , the difference between that score’s rank and the winning score’s rank. The bottom bar chart of Fig. 2 shows the median for each score; the smallest median was − 4 for both Multiscore and word2vec GDx.

Fig. 2.

Analysis of subscores within Multiscore. Top: Stacked bar plot representation of the tally of the number of cases for which that score had the highest rank for the diagnostic gene. Blue = number of cases where a score “tied” (tied for highest prioritization of positive gene), orange = number of cases where this score “won” (outright highest prioritization of positive gene). Bottom: ; median difference between the score and the winner, for cases where the score was not the winner. Red = median rank difference from winner, recorded when a score is not the “winner.”

As the GDx positive case count of the diagnostic gene (cc) increased, so did the predictive ability of Multiscore (Table 3). Genes with cc = 0 have not been reported as positive results previously, thus there is no phenotypic information in the GDx knowledge dataset, and the Multiscore model relies on literature and OMIM knowledge datasets.

Table 3.

Average recall at k of multiscore for all cases based on GDx positive case count of the diagnostic gene (cc)

k	All cases	cc = 0	cc = 1	cc = 2–5	cc = 6–10	cc ≥ 11
1	33.1%	2.3%	8.0%	14.6%	21.1%	38.4%
2	47.9%	4.6%	13.3%	25.0%	33.3%	54.8%
3	57.2%	8.3%	18.9%	30.8%	41.8%	64.8%
4	63.9%	11.1%	22.5%	36.4%	48.9%	71.9%
5	69.1%	13.4%	26.1%	42.4%	55.2%	76.9%
10	83.5%	21.7%	46.2%	61.4%	72.5%	90.4%
15	89.8%	30.4%	57.4%	75.8%	83.2%	95.1%
20	93.3%	40.6%	67.1%	83.9%	90.6%	97.1%
n cases	9,989	217	249	948	881	7,694

“n cases” refers to the number of tested cases with the relevant cc values

Multiscore can prioritize genes that are not curated in public reference databases

The Multiscore testing set included 74 genes without HPOA information, impacting 257 cases (Table S5). In 231/257 (90%) cases, the positive gene had been previously seen in the GDx dataset (61 genes). Data in the GDx and literature datasets brought the Multiscore phenotypic rank within the 20 top genes in 48.2% of the cases (recall of 0.482 at k = 20) allowing prioritization of newer and ultra rare disease associations. One of these genes, KDM2B, did not have a curated entry in OMIM (Amberger et al. 2009), Orphanet (Aymé 2003), or GeneReviews, it had however one large publication reporting the phenotypic spectrum in 27 individuals (van Jaarsveld et al. 2023) and was included in the GeneDx knowledge reference; Multiscore ranked the KDM2B gene within the top 20 genes in 4/8 cases.

Multiscore outperforms other phenotype-only prioritization tools

Multiscore showed higher average recall compared to Phrank GDx, Phrank HPOA (Jagadeesh et al. 2019), and LIRICAL (Robinson et al. 2020) (Table 4 and Figure S3). At each k value, the next-best performer was Phrank GDx; Phrank HPOA (Jagadeesh et al. 2019) and LIRICAL (Robinson et al. 2020) average recall scores were lower and within 4% of each other for all tested k values.

Table 4.

Ranking performance of multiscore, phrank (Jagadeesh et al. 2019) with GDx GPA knowledgebase (Phrank GDx), phrank with HPOA GPA knowledgebase (Phrank HPOA, Jagadeesh et al. 2019), and LIRICAL (Robinson et al. 2020)

Average recall at k	Multiscore	Phrank GDx	Phrank HPOA	LIRICAL HPOA
1	33.1%	25.8%	23.1%	22.5%
2	47.9%	39.2%	34.2%	34.1%
3	57.2%	49.2%	41.9%	42.0%
5	69.1%	62.3%	51.9%	53.5%
10	83.5%	78.8%	67.3%	69.6%
15	89.8%	86.1%	76.2%	78.2%
20	93.3%	90.4%	81.8%	83.9%
Median rank	3	4	5	5
Mean rank	6.35	10.8	15.4	14
Rank standard deviation	10.2	35.6	38.7	35.0

Average recall at k, median rank, mean rank, and rank standard deviation all refer to prioritization of the positive reported gene

The overall wins tally of Multiscore and each algorithm is shown in the top bar chart of Figures S4 (all cases) and S5 (cc < 11). Multiscore gave the highest ranking for the positive finding in 2,432 cases and tied for highest in an additional 2,942 cases, for a total of 5,374/9,989 (53.8%) cases tested, followed by Phrank GDx, which was the highest-ranking score in 1,955 cases and tied for highest in 2,349 cases, for a total of 4,304/9,989 (43.1%) cases. Multiscore showed the smallest median (see above) at -3.

When only cases with cc < 11 are considered (n = 2,295, Figure S5), Multiscore retains the highest performance (number of outright wins plus ties), with the highest rank in 691 cases and tied for highest in 354 cases, for a total of 1,045/2,295 (45.5%) cases. Phrank GDx, which relies solely on the GDx GPA knowledgebase, drops to the lowest number of outright wins plus ties out of the four benchmarked algorithms, with the highest rank in 283 cases and tied for highest in 148 cases, for a total of only 431/2,295 (18.8%) cases, showing the advantage of Multiscore leveraging multiple knowledgebases.

A prioritization tool that uses a single GPA knowledgebase cannot prioritize a missing gene. In 217 cases without GDx GPA data and 257 cases without HPOA GPA data for the positive gene, Phrank GDx and Phrank HPOA (Jagadeesh et al. 2019) provided no prioritization, respectively. LIRICAL (Robinson et al. 2020) includes ORPHANET (Aymé 2003) disease entries which added phenotype annotations for the positive gene in an additional 30 cases.

Multiscore can handle real world patients

Clinical information for the knowledge reference and the testing set is abstracted from clinical notes of a diverse dataset (Table 1) without curation. The median number of HPO terms per case in the test set was 15. Although a linear regression revealed only a weak correlation (Pearson’s R = -0.20), the rank of the positive gene decreased by 0.23 for each HPO term describing the patient. Multiscore performance remained high as demonstrated by recall at k (Figure S6). As more cases are assigned to a given gene, the GPA set incorporates “random” phenotypes not directly related to the positive finding in a case. These phenotypes, although rare in the patients assigned to a gene, are common and nonspecific across other disease genes; in other words, those GPAs have a low TF-iDF for the given gene. Figure S7 contains histogram data on the median TF-iDF GPA for genes in the GDx dataset.

Discussion

Uncoupling genotype from phenotypic fit allows analysts to consider molecularly strong findings that could represent phenotypic expansions or completely new disease associations; however, for known diseases, accurate phenotype assessment remains paramount. Existing algorithms measure phenotypic similarity against a curated reference set of Mendelian conditions that need expert and often manual curation (Amberger et al. 2009; Aymé 2003; Cornish et al. 2018; Javed et al. 2014; Yuan et al. 2022), often rendering them outdated. In contrast, the phenotypes in individuals with a positive finding identified in clinical practice provide insights that can guide future molecular diagnoses, underscoring the importance of leveraging our internal knowledgebase.

The importance of the GPA knowledgebase is illustrated by the ability of Multiscore to identify matches for gene-disease associations not yet curated in OMIM, which allowed the diagnosis in 2.3% of cases in this cohort, and by its improved performance when compared to other prioritization tools – Phrank GDx and Phrank HPOA (Jagadeesh et al. 2019). The fact that the GDx HRSS and Jaccard subscores overtake their Lit dataset counterparts at higher values of k also suggests that the GDx GPAs describe a more comprehensive picture of the phenotypic spectrum of gene disease associations. Although future work is still needed to address phenotypic noise as our dynamic dataset continues to grow, Multiscore performance remained consistent despite incorporating low TF-iDF terms, since it combines both curated datasets and real-world data to create the knowledge reference.

Multiscore performance was impacted by low case counts for the positive gene, which can be a problem for new disease associations and is a limitation of the current model. However, one advantage of Multiscore’s ensemble architecture is that it enables iterative self-improvement by adding data sources to our knowledgebase or additional algorithms as subscores. Knowledge graphs and other architectures that explore disease space through protein-protein interactions, molecular functions, and biological networks, as in PPAR (Gnanaolivu et al. 2025), SHEPHERD (Alsentzer et al. 2025), Phenolyzer (Yang et al. 2015), and phen2gene (Zhao et al. 2020) or transformer-based language model embeddings (Huang et al. 2020 10.48550/arXiv.1904.05342, Lee et al. 2019 10.48550/arXiv.1901.08746, Gu et al. 2021 10.48550/arXiv.2007.15779), could be explored to potentially capture nuanced gene-phenotype representations that may reside within their contextual frameworks.

In the clinical laboratory, both the genotype filters and the phenotype match play a role in identifying diagnostic findings, i.e. a match between a phenotype fit in the tested individual and the knowledge reference for a gene is only significant if there are reportable variants in that gene. Multiscore has been implemented as an additional tool to highlight genes that should be more carefully investigated even if they don’t stand out in genotype-driven searches, e.g., variants present in population controls or single heterozygous variants in autosomal recessive genes could have a high Multiscore if the gene matches the patient’s phenotype. Certain circumstances allow more variants to pass the genotype filters, for example, in non-trio cases, where any variant not seen before could potentially be associated with the phenotype as a de novo variant, or any two variants in a gene could potentially be compound heterozygous. This allows more genes for Multiscore to rank (285 compared to 84 in trio cases), and the median rank of the diagnostic finding is 4 compared to 2 in trio cases. In practice, the outcome is additional genes are highlighted as strong phenotypic fit and can be returned to patients and providers to allow follow up testing and clinical correlation to identify the true diagnostic finding. However, on average, less than half of the genes in the genotype searches were identified as a potential phenotypic match by Multiscore. It is expected that Multiscore might also aid clinical review by incorporating the phenotypes of previously tested individuals which would otherwise be time consuming to recognize, however the exact impact on time savings would require additional investigation and was out of scope for this validation study.

In conclusion, Multiscore is a validated, ensemble-based gene prioritization tool that allows for the identification of the positive gene in the top ten genes in 84% of the test cases. The key to the performance of Multiscore is related to the rich and diverse information provided by our internal database, demonstrating that phenotypes of individuals in the clinic are required to fully inform the phenotypic presentation of associated diseases. This robust approach to gene prioritization helps scale case throughput responsibly, increasing the likelihood of timely diagnoses. Multiscore represents the first step towards a future holistic ranker that incorporates variant-specific features.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Vincent D. Ustach and Maria J. Guillen Sacoto. The first draft of the manuscript was written by Vincent D. Ustach and Maria J. Guillen Sacoto and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Data availability

Code to reproduce the analysis and the patient dataset used to test and validate it are proprietary and have not been deposited in a public repository.

Declarations

Conflict of interest

V.D.U., M.J.G.S., S.M., A.B., F.M.F., M.G., H.G., K.Mc., K.M., G.R., N.T., B.J. and T.L. are salaried employees of and may own stock in GeneDx LLC. V.G., K.A., K.R., R.T., and F.M.Z. were salaried employees of GeneDx LLC at the time Multiscore was developed.

Ethical approval

This study was performed in line with the principles of the Declaration of Helsinki. This study, using deidentified data, was conducted under a GeneDx research protocol approved by the Western Institutional Review Board, Study Number 1169768, WIRB Pro Number 20162523, which states that this research meets the requirements for a waiver of consent.

Consent to participate

See above.

Consent to publish

Not applicable.