Combining multiplexed functional data to improve variant classification

Abstract

Background

With the surge in the number of variants of uncertain significance (VUS) reported in ClinVar in recent years, there is an imperative to resolve VUS at scale. Multiplexed assays of variant effect (MAVEs), which allow the functional consequence of 100s to 1000s of genetic variants to be measured in a single experiment, are emerging as a powerful source of evidence which can be used in clinical variant classification. Increasingly, multiple published MAVEs are available for the same gene, sometimes measuring different aspects of variant impact. When multiple functional roles of a gene need to be considered, combining data from multiple MAVEs may provide a more comprehensive measure of the consequence of a genetic variant, which could impact variant classifications.

Methods

We curated published datasets from five MAVEs for the gene TP53, two MAVEs for LDLR and two MAVEs for PTEN. Statistical methods (principal component analysis), unsupervised learning (k-means clustering), and supervised learning (Naïve Bayes and random forest classifiers) were used to integrate multiple MAVE datasets. The utility of MAVE integration methods were assessed using standard metrics (sensitivity, specificity, etc) as well as evidence strength in a putative variant classification framework.

Results

Here, we provide guidance for combining such multiplexed functional data, incorporating a stepwise process from data curation and collection to model generation and validation. We also present a web applet that allows users to test various methods for combining score sets from multiple assays, calculate integrated functional scores for all variants, and assess whether combining data enables the application of stronger evidence for pathogenicity or benignity. In general, supervised learning methods such as random forest led to improved variant classification as compared to any individual MAVE dataset.

Conclusions

By following the steps outlined herein with appropriate guardrails, researchers can maximize the value of MAVEs, strengthen the functional evidence for clinical variant classification, and potentially uncover novel mechanisms of pathogenicity for clinically relevant genes.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s13073-026-01715-w.

Background

Next generation sequencing is widely used clinically to provide a molecular diagnosis for genetic diseases. However, it is difficult to predict the functional impact, and therefore disease risk, associated with rare variants in Mendelian disease genes. We often lack sufficient information to classify these variants as pathogenic or benign and they become variants of uncertain significance (VUS). This is especially true for rare missense variants, > 90% of which are currently classified as VUS in ClinVar [1]. The high likelihood that a newly observed variant will be a VUS has made interpretation of genetic variants a substantial bottleneck in clinical genetics; absence of a causative pathogenic variant, and thereby a molecular diagnosis, limits effective counselling, management and treatment, restricting patient access to prophylactic, pharmaceutical and, looking to the future, potential gene therapy interventions. Variant interpretation is particularly challenging where other lines of evidence (de novo status, allele segregation, family history, etc.) are not available. Furthermore, interpretation of variants identified in individuals of non-European-like genetic ancestries is often limited by the poor diversity of our clinical and population databases, causing substantial inequity in diagnosis and treatment [2].

Experimentally derived variant effect data provide a valuable source of information to assist in variant classification. According to the American College of Medical Genetics and the Association for Molecular Pathology (ACMG-AMP) guidelines for interpretation of sequence variants, data from “well-established” functional assays can provide strong evidence towards pathogenicity and benignity for variant classification under the PS3 and BS3 criteria, respectively [3]. However, reactive studies that model the function of individual variants cannot match the scale at which novel variants are discovered and are prohibitively laborious and expensive. Therefore, proactive, large-scale investigations of genetic variant effects are much needed.

As nucleic acid synthesis and sequencing have become more economical and accessible, a new generation of assays called multiplexed assays of variant effects (MAVEs) has evolved. MAVEs systematically assess the functional impact of thousands of variants, typically all single-nucleotide variants (SNVs) or all amino acid substitutions, in a given gene or functional domain [4]. While many MAVEs use pooled in vitro mutagenesis and introduction of an exogenous variant library into cells to assess function, approaches employing CRISPR/Cas9-based genome engineering such as Saturation Genome Editing [5–12], base editors [13–16], or prime editors [17, 18] are being increasingly employed to enable systematic editing of endogenous gene loci [19]. This approach allows splice-modulating and other non-coding variant effects to be tested and enables variant effect assessment informed by the native sequence context.

Compared to traditional functional assays of individual variants, MAVEs offer substantial time and cost savings per-variant. In addition, the effect of an individual variant is interpreted in the context of all other possible variants at the target locus for the tested molecular phenotype. Hence, MAVEs can allow rigorous statistical analysis, including estimation of the assay sensitivity and specificity, and positive (PPV) and negative predictive value (NPV), enabling appropriate weighting of the information within diagnostic variant classification frameworks [20, 21]. MAVEs can be designed to measure either loss-of-function (LOF) or gain-of-function (GOF) of a gene and thereby matched appropriately to the known genetic etiology of a corresponding genetic disorder. MAVEs leverage the curation of missense truth-set variants, or variants of known benignity or pathogenicity, enabling MAVE data to be used in variant classification frameworks such as the widely used ACMG/AMP framework. By assessing all variants in a gene, variant effect sub-groups can be identified, facilitating genotype-phenotype correlations which may be prognostically informative [22].

The last decade has seen an explosion of MAVEs, measuring millions of variant effects and using different modalities to study variants across a variety of clinically important genes [23]. In fact, multiple MAVE datasets (hereafter referred to as multiplexed functional data) are now available for a number of genes (Table 1). The next decade is likely to see an increasing number of genes for which two or more multiplexed functional datasets are available. For some genes, such as TP53, guidance on using functional data in clinical variant interpretation is available through a gene-specific ClinGen Variant Curation Expert Panel (VCEP) [24]. However, for most other genes, such guidance is unavailable, leading to confusion and non-standardized practices in how MAVE data are used across different clinical testing laboratories. In these cases, combining experimental output data from two or more MAVEs may increase overall accuracy and clinical utility. Most simplistically, the impact of experimental noise may be mitigated by combining measurements across multiple assays. Furthermore, there are specific advantages to combining data from multiple phenotypic readouts or cell types, enabling a more comprehensive interpretation of variant effects.

Table 1.

Exemplar genes with two or more multiplex functional datasets currently available

List of genes with multiple MAVE datasets currently available
Gene REF(s)	Genetic disorder	Missense variants in ClinVar (B/LB, P/LP, VUS)	MAVE assay types
ASPA [25]	Spongy degeneration of central nervous system	7; 73; 62	Abundance, cell fitness
BRAF [26]	RASopathy	13; 116; 444	LABEL-seq (activity, protein-protein interactions and druggability)
BRCA1 [6]	Hereditary breast ovarian cancer syndrome	384; 266; 1,552	Cisplatin resistance, homology-directed repair activity
BRCA2 [27, 28]	Hereditary breast ovarian cancer syndrome	369; 96; 3,596	Cell fitness, homology-directed repair activity
CARD11 [8]	Severe combined immunodeficiency due to CARD11 deficiency	80; 17; 340	Cell fitness with different small molecule treatment
CCR5* [29]	Virus susceptibility	3; 0; 29	Cell surface abundance, gp120 binding
CHEK2 [30, 31]	Hereditary cancer-predisposing syndrome	9; 12; 1,651	Cell fitness
CXCR4* [29]	Warts, hypogammaglobulinemia, infections, and myelokathexis	19; 1; 87	Cell surface abundance, CXCL12 binding
F9 [32]	Hereditary factor IX deficiency disease	25; 186; 66	MultiSTEP secretion assay with multiple antibody readouts
FGFR1 [33] FGFR2 FGFR3 FGFR4	Oncogene, FGFR inhibitor resistance, others	23; 89; 377 11; 101; 295 61; 54; 427 8; 0; 99	Activity, FGFR-inhibitor resistance
GCK [34, 35]	Monogenic diabetes	12; 335; 217	Abundance, activity
HRAS [36]	RASopathy	54; 35; 226	Cell fitness with or without GAP/GEF regulators
KCNE1 [37]	Long QT Syndrome	; 6; 108	Cell surface abundance, activity
KCNH2 [38]	Long QT Syndrome	44; 179; 1,326	Cell surface abundance, activity
KCNQ1 [39]	Long QT Syndrome	19; 235; 637	Cell surface abundance, activity, dominant negative
LDLR [40, 41]	Familial Hypercholesterolemia	44; 739; 777	Cell surface abundance, activity
LMNA [42]	Cardiomyopathy, Hutchinson-Gilford progeria, others	24; 225; 761	VIS-seq (variant abundance, localization, cell morphology)
NUDT15 [43–45]	Poor metabolism of thiopurines	1; 0; 6	Abundance, thiopurine cytotoxicity
POLG [46]	Progressive sclerosing poliodystrophy, others	30; 120; 1,251	Cell fitness, dominant negative
PRKN [47, 48]	Parkinson’s disease	15; 19; 154	Mitophagy, phosphorylated ubiquitin abundance
PTEN [42, 49, 50]	Hereditary breast ovarian cancer syndrome	26; 277; 742	Abundance, activity, VIS-seq
SNCA [51]	Neurodegenerative disease including Parkinson’s	3; 5; 28	Abundance, cell fitness
SOD1 [52]	Amyotrophic lateral sclerosis	0; 112; 51	Abundance, cell fitness
TARDBP [53, 54]	Amyotrophic lateral sclerosis type 10	9; 28; 87	Abundance, cell fitness
TP53 [55–57]	Li-Fraumeni syndrome	144; 336; 609	Cell fitness, dominant negative
VKORC1 [58]	vitamin K-dependent clotting factor deficiency, warfarin response	6; 5; 12	Abundance, activity
Example genes where a single MAVE study contains multiple scoresets$
DDX3X [10]	Intellectual disability, X-linked 102	5; 124; 172	Cell fitness (5 timepoints)
RAD51C [9]	Hereditary cancer-predisposing syndrome	68; 17; 608	Cell fitness (3 timepoints)
BAP1 [12]	Tumor predisposition syndrome	23; 27; 1,171	Cell fitness (5 timepoints)
VHL [5]	Von Hippel-Lindau syndrome	72; 176; 545	Cell fitness (3 timepoints)

MAVEregistry (https://registry.varianteffect.org/) and MaveDB (https://mavedb.org/) were accessed on 26 Feb 2026 to generate this list. Additional genes were manually curated. The ClinVar database was accessed 22 Jan 2025 to obtain the number of variants classified as B/LB, P/LP, or VUS [59]

*CXCR4 and CCR5 are also key proteins bound by HIV-1 as part of viral entry into the cell; CCR5 genetic variants also regulate susceptibility to other viruses, including West Nile virus and Hepatitis C

^$It is not practical to generate an exhaustive list of genes that meet this criteria, but example genes are highlighted. Most of these fit the most common case of timeseries of cell fitness data with multiple time points

While integrated analyses have been published for specific genes [60–62], systematic guidance for when and how to combine MAVEs has been lacking. Here, we discuss two scenarios in which combining variant effect data across MAVEs may be beneficial and, conversely, describe scenarios where combining such multiplexed functional data would not be recommended. We provide a set of recommendations and best practices for selecting and combining multiplexed functional data to improve their clinical utility in variant classification. Finally, we provide three practical examples where different methods are used to integrate multiplexed functional data from five MAVEs for the gene TP53, two MAVEs for LDLR, and two MAVEs for PTEN.

Methods

Our recommendations were developed through a collaborative, consensus-based process, and are illustrated here using three gene examples: TP53, LDLR, and PTEN.

Development of recommendations

The recommendations presented here grew out of discussions among members of the Clinical Variant Interpretation (CVI) Working Group within the Atlas of Variant Effects (AVE) Alliance, a group of researchers and clinicians generating or applying multiplexed functional data worldwide. An initial set of recommendations developed by a subgroup was presented at multiple CVI working group meetings to further refine specific recommendations. Subsequent iterations of the recommendations involved conversations among expert clinicians, ClinGen leaders, and MAVE experts before agreement on the final content of the recommendations was reached.

Curation of clinical truth sets for TP53, LDLR, and PTEN

A major concern when working with well-studied genes such as TP53 is the possibility of circularity in truth sets, where variants classified by a given functional assay are used as “ground truth” to calibrate and validate the same or highly similar functional datasets. To avoid this circularity and maintain evidence independence, we used a clinical truth set curated to only contain TP53 variants that could be classified in the absence of functional and computational data [63]. This dataset was constructed by Fortuño et al. [63], in which TP53 variants were assigned likelihood ratios (LRs) based on clinical data, functional assays, and bioinformatic tools. These LRs were combined to calculate a posterior probability, which was then used to assign variant classifications. We modified this dataset by removing all LRs from functional datasets and bioinformatic tools. We then applied a naive prior probability of pathogenicity of 0.1 and recalculated the posterior probability using only the remaining clinical LRs. Variants were reclassified based on the following posterior probability thresholds: Pathogenic (> 0.99), Likely pathogenic (0.90–0.99), Likely Benign (0.001–0.010), Benign (< 0.001) [64]. The new classifications were then used as an updated truth set for downstream analyses. The LDLR clinical truth set was identical to that used in the LDLR MAVE publication and consisted of benign or likely benign (B/LB) and pathogenic or likely pathogenic (P/LP) missense variants retrieved from ClinVar [40]. The PTEN clinical truth set was similarly curated by retrieving B/LB and P/LP missense variants from ClinVar (accessed 11 Jun 2025).

Circularity considerations for LDLR and PTEN truth sets

When feasible, we adjusted for circularity by excluding functional and computational evidence from truth set construction, as described above for TP53. For LDLR, the truth set is identical to that used in the Tabet et al. MAVE publication [32], which was constructed from ClinVar variants classified prior to publication of those MAVE data. The degree of circularity for the LDLR truth set is therefore expected to be lower than for TP53, though we acknowledge that some ClinVar submitters may have incorporated functional evidence from published experiments preceding the publication of the LDLR MAVE in their classifications. For PTEN, a ClinVar-derived truth set was used directly. As noted in the Results, some PTEN ClinVar classifications incorporated functional data, potentially introducing circularity and bias into downstream analyses; this is discussed as a limitation of the PTEN example.

Development of web Shiny application and demonstration of recommendations using TP53, LDLR, and PTEN data

A Shiny web application was developed to demonstrate the integration of a multiplexed functional dataset herein and to facilitate its sharing with the broader scientific community. Input datasets were curated, with each row representing a different variant and each column the average functional score for one input MAVE [55, 56]. For the purposes of this demonstration, only variants scored by all assays were included. The clinical truth sets derived above were supplemented with different fractions of nonsense and synonymous variants, or “proxy” truth set variants, depending on the size of the clinical truth set. For TP53, which has a robust clinical truth set, no nonsense or synonymous variants were included in model training. For LDLR, the clinical truth set was supplemented with a random selection of 60% of available synonymous and premature termination codon (PTC) variants for model training. For PTEN, the full set of synonymous and nonsense variants was included in the training set. For both LDLR and PTEN, PTCs in the last exon and the last 50 nt of the penultimate exon, which are known to escape nonsense-mediated RNA decay, were filtered out as they may not induce LOF. See “Implementation of integration methods for illustrative examples” below for details of statistical methods, software packages, and model parameters used.

For TP53, supervised learning methods used a randomized 70%/30% train/test split, while 50%/50% and 40%/60% train/test splits were used for LDLR and PTEN, respectively. Train/test split fractions were determined by truth set size: larger truth sets allow more data to be allocated to training while still retaining a sufficiently large test set for reliable performance estimation. Specifically, the 70%/30% split was used for TP53 because its truth set (201 missense variants) is large enough to support a 30% held-out test set of 61 variants, sufficient for stable metric estimation. A 50%/50% split was used for LDLR to balance training and test sets given its intermediate-sized and highly imbalanced truth set (15 B/LB and 356 P/LP variants). A 40%/60% split was used for PTEN because its small number of B/LB variants (16 in the merged dataset) necessitated a larger test fraction to obtain a reliable estimate of classifier performance on unseen data.

For k-means clustering, the output column is a categorical variable indicating the cluster; for other combining methods, the output column is a single integrated score. Model metrics are computed using a single threshold, either determined by ROC analysis to best distinguish B/LB variants from P/LP variants or set manually by inspecting the separation of B/LB and P/LP variants in a histogram of functional scores. Evidence strength was determined using the odds of pathogenicity (OddsPath) framework described by Brnich et al. [20], which uses Bayesian statistics to convert functional assay performance into likelihood ratios for clinical variant classification within the ACMG/AMP framework. All the model metrics, threshold values used by various classifiers and variant-level integrated scores assigned by different methods are downloadable as tables through the Shiny application.

Implementation of integration methods for illustrative examples

Four integration methods were applied to the illustrative examples in this paper: (1) principal component analysis (PCA); (2) unsupervised k-means clustering; (3) a supervised naive Bayesian classifier; and (4) a supervised random forest classifier. All analyses were performed in R (4.2.3). PCA and k-means clustering were implemented using the built-in stats package. To compute model performance based on PCA, the first principal component (PC1) was used as a measure of integration of variant-effect data. K-means clustering used 4 clusters with a set size of 20, and the categorical variable output was used to calculate model performance. The caret package (https://topepo.github.io/caret/) was used to train the naive Bayesian classifiers with default hyperparameters. The caret package was also used to train the random forest classifiers with five trees and the default hyperparameters. The pROC package (https://xrobin.github.io/pROC/) was used to compute model performance (sensitivity, specificity, etc.). ggplot2, ggbiplot, and ggridges packages were used to generate plots for OddsPath, unsupervised learning, and supervised learning, respectively. Ellipses present in plots generated by ggbiplot (scatterplots of principal components 1 and 2) were based on either known class or k-means clustering with a data ellipse probability of 0.68. The shiny and bslib packages were used to generate the web Shiny app hosted at shinyapps.io. Additional packages used for data frame manipulation, saving SVG files, etc., are listed at the top of the available Shiny web app script (https://github.com/jagannathan-lab/2025-calhoun_et_al/blob/main/AVEshinyRcode_v2p99_mar2026.txt).

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work, the authors used Claude.ai to improve readability, flow, and language. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.

Results

With the above context in mind, we now turn to practical guidance for when and how to combine multiplexed functional datasets. We will focus on two scenarios in which combining multiplexed functional data may be beneficial.

1. One gene, different MAVEs: A genetic condition may be caused by multiple mechanisms, such as loss-of-function (LOF) and dominant-negative (DN) gain-of-function. However, some MAVEs may only be capable of measuring either the LOF mechanism or the DN, but not both. Even within the LOF mechanism, many MAVEs measure only one aspect of protein function. For example, variant abundance by massively parallel sequencing (VAMP-seq) assays measure protein stability but are blind to enzymatic activity [49]. Thus, despite the often excellent positive predictive value (PPV) for identifying a pathogenic variant, the ability to confidently determine a variant as benign (negative predictive value) in such functional assays is impaired [6, 60]. In such cases, combining multiplexed functional data measuring multiple aspects of protein function to generate a single integrated score may improve the accuracy of functional predictions for that gene.

2. One MAVE, many measurement sets: Within a MAVE, the dynamic range can be extended by collecting data at multiple time points to provide quantitative information on a phenotype that may improve variant classification [9, 10, 12]. A similar argument could be made for MAVEs that measure the same molecular phenotype but through different assays, in different model systems and cell lines, or at different concentrations of a drug or other treatment that may distinguish different degrees of gene function. By quantifying variant effects over time or across different experimental conditions, putative hypomorphic variants that cause partial functional deficits could be potentially distinguished from complete LOF variants.

When not to combine multiplexed functional data

There are scenarios in which combining multiplexed functional data may not be desirable. For example, distinct disorders can arise through different mechanisms of genetic variants within the same gene (i.e. distinct gene-disease dyads [65]). In the case of the CARD11 gene, gain-of-function, loss-of-function, and dominant negative variants result in distinct genetic disorders [66]. As a measure of functional variant effect for use in clinical variant classification, separate scores and/or thresholds may be required for each gene-disease dyad. It is also important to consider technical aspects of MAVE design which determine the type of variants that can be assessed. Some MAVEs are based on a transgenic cDNA platform, whereas others that use genome editing of an endogenous allele are “splice-aware”. Care must be taken when integrating data to ensure that results from a cDNA assay do not override those from a genome-editing assay for variants that affect splicing.

Considerations

Hypothetically, a single MAVE could distinguish every P/LP variant from every B/LB variant for a gene-disease dyad, presuming there is a single mechanism of pathogenicity. In practice, there is usually overlap between functional scores for some known functionally normal and functionally abnormal variants (Fig. 1A-B). Despite this overlap, it is usually possible, at least for loss-of-function (LOF) assays, to set thresholds that adequately distinguish the majority of synonymous variants from nonsense variants, as well as variants previously classified clinically as B/LB from those classified as P/LP. There are several ways to assess assay performance at this stage: (1) the dynamic range to separate functionally normal and abnormal variants, and (2) computing standard classification metrics (sensitivity, specificity, PPV, NPV, etc. shown in Fig. 1C). These metrics establish a baseline for each dataset, which is critical for later investigation of how combining multiplexed functional data affects them.

Fig. 1.

Assessment of assay performance prior to combining multiplexed functional data. A Functional score distributions for a hypothetical perfect multiplexed functional assay for assumed functionally normal (synonymous) & B/LB variants and assumed functionally abnormal (premature termination codon, i.e. PTC) & P/LP variants. B An example of real-world multiplexed functional data illustrating overlap between the P/LP and B/LB variants scored in the assay [55]. C Critical metrics for assessing MAVE performance

Clinical genetic diagnostics currently uses a specific framework outlined in 2015 by the ACMG/AMP to robustly ascribe pathogenicity or benignity to individual variants [3]. In this framework, different sources of evidence are applied at varying strengths and combined to reach a final classification. Functional scores from a MAVE can currently be incorporated as one source of evidence via the PS3 and BS3 criteria (towards pathogenicity and benignity, respectively) in the ACMG/AMP framework. The strength of the applicable evidence is determined using the Bayesian framework described by Brnich et al. [20] and depends on assay performance metrics calculated from a clinically validated truth set. The Brnich framework calculates likelihood ratios (termed “OddsPath”) by comparing the prior probability of pathogenicity in the truth set (proportion of P/LP variants in the truth set) with the posterior probability of pathogenicity among variants falling above or below a defined threshold for functionality (proportion of P/LP variants among those categorized as functionally abnormal or normal). The prior and posterior probabilities are then used to calculate assay-specific likelihood ratios (OddsPath) towards pathogenicity and benignity [20]. These likelihood ratios can be calculated for both individual multiplexed functional assays and integrated scores to assess whether data integration improves evidence strength and enables application of stronger evidence categories within the functional evidence range.

To combine multiplexed functional data to generate an integrated score that can provide stronger evidence to improve clinical variant classification, some basic criteria must be met:

1. The relationship between what the assay measures and the human phenotype are the same for both assays. To combine different multiplexed functional data for a particular gene-disease dyad, it is important to assess whether the molecular phenotypes of each MAVE being combined are relevant to the disease and overall gene-disease dyad, even if the precise molecular mechanisms are not fully understood. For example, one cell growth-based CARD11 MAVE measured a LOF phenotype linked to immunodeficiency disorders such as severe combined immunodeficiency (SCID) [8]. Conversely, measuring the response of cells after treatment with the BTK inhibitor ibrutinib enabled the same cell-based system to assess a GOF phenotype linked to the lymphoproliferative disorder BENTA [8]. Therefore, the assays are measuring variant effects relevant to different clinical phenotypes and likely distinct molecular mechanisms and should be assessed independently rather than combined.

2. The multiplexed functional data being combined provide complementary information. Combining datasets generated by the same assay performed in two different laboratories may yield some benefit in providing a better estimate of the true functional effect across more measurements, but the data are likely to be highly correlated and additional information from the integrated score may be limited. Conversely, two or more assays which measure different aspects of molecular function associated with the same clinical phenotype may be more likely to improve variant classification outcomes when combined by providing a more complete picture of overall protein function, even when the assays show some correlation. For instance, independent assays have been developed quantifying the impact of PTEN variants on both protein abundance and thiopurine toxicity, which has been used as an independent readout to investigate the impact of NUDT15 variants [45]. Work by Cagiada et al. has shown that around half of the variants leading to loss-of-function in these assays also reduce protein abundance [43]. By combining multiplexed functional data for these genes, abundance might therefore serve as a complementary readout, increasing confidence in constituent predictions of pathogenicity/benignity for a subset of loss-of-function variants [43].

3. A sufficient number of truth set variants are available for assay validation [20]. Construction of truth sets for combined multiplexed datasets may not be feasible where the loci targeted by each dataset are distinct, e.g., when constituent MAVEs target specific functional domains. In the absence of a robust clinical truth set, it may be possible to validate based on synonymous (presumed benign) and nonsense (presumed pathogenic) variants if the disease mechanism is loss of gene function. However, the strength of evidence that can be applied in the ACMG/AMP framework is likely to be limited without a sufficiently large clinically relevant truth set.

We acknowledge that criteria 1 and 2 are not always easy to assess. In that case, we recommend being guided by the combined score’s ability to accurately categorize truth-set variants as normal or abnormal relative to the individual MAVEs.

Practical considerations for integrating multiplexed functional data to generate an integrated score

A general workflow for integrating multiplexed functional data comprises: (i) Collecting, harmonizing, and merging multiplexed functional data and truth set variants; (ii) Choosing an appropriate method to generate an integrated score; (iii) Setting thresholds for functionally normal and abnormal variants; and (iv) Final assessment of whether the integrated score has improved utility for variant categorization (Fig. 2A).

Fig. 2.

Suggested pipeline and decision tree to generate and evaluate integrated functional score. A A method for combining data will involve merging the datasets from different MAVEs and the truth set. A final score for each variant will be generated using an appropriate supervised or unsupervised method. Threshold values to separate functionally normal and abnormal variants will be set based on truth set. After thresholding, the integrated score must be assessed for improved variant resolution compared to the multiplexed functional data from individual MAVEs based on how well the different scores distinguish B/LB and P/LP variants in the truth set. B Decision tree approach to choosing an appropriate method to combine multiplexed functional data

• i)Collecting, harmonizing, and merging multiplexed functional data and truth set

Prior to merging datasets, it is important to ensure that each dataset uses standardized variant nomenclature, such as the Human Genome Variation Society (HGVS) nomenclature, and that each dataset references the same canonical gene transcript(s) before merging. Tools such as Transvar can be used to harmonize data, ensuring that variants are matched to a common transcript [67]. Next, decide if only variants evaluated across all individual MAVEs will be retained or if missing data will be allowed. Relatedly, decide whether any variants should be excluded from specific MAVEs, such as those predicted to affect splicing from cDNA-based assays. See Section iii for more guidance on these considerations. Lastly, one may wish to transform the data prior to modeling. For example, standardization, or scaling each input dataset to a common scale (typically 0–1), is commonly employed prior to supervised learning methods. While linear rescaling is most commonly applied, depending on the specific context of the MAVE, a non-linear rescaling method may be most appropriate. Prior to standardization, it can be useful to normalize (e.g., via a log or other transformation), particularly if the dataset is not normally distributed.

Truth set variants will be used both for model training (supervised learning) and to evaluate how well the final integrated score performs at categorizing variants of known pathogenicity and benignity as abnormal and normal, respectively. It is highly recommended, when feasible, to collect truth set variants from an appropriate clinical database, such as ClinVar [68] or LOVD [69], while keeping in mind that the classification of some of these variants may have relied on one or more of the functional assays being evaluated, introducing circularity. We recommend manually curating variants to only retain those that were not classified using functional evidence (as we do for the TP53 example). Where this is not possible, one could obtain a historical truth set from an archived version of a clinical database from a time prior to the release of the functional data, although doing so will likely reduce the size of the truth set considerably.

When clinically validated truth sets of sufficient size are unavailable, as is often the case, synonymous (presumed benign) and nonsense (presumed pathogenic) variants included in a multiplexed functional assay can serve as a proxy truth set if the disease mechanism is loss-of-function [6, 50, 60]. There are key caveats to using such a proxy truth set, including the loss of dynamic range in ascertaining B/LB or P/LP variants that are partially functional as well as misannotation of certain nonsense variants as P/LP when they are truly benign [70] or synonymous variants as B/LB when they may induce missplicing of a transcript [71]. Pre-emptive filtering of synonymous variants predicted to impact splicing and nonsense variants that occur in the last exon or within 50 nucleotides of the last exon-exon junction [72, 73] from proxy truth sets is recommended in such cases. Finally, while the OddsPath framework can be applied to either clinically ascertained truth sets or proxy truth sets (or a combination), robust evidence strength assignment and clinical utility require clinically validated truth sets.

For dominant-negative (DN) mechanisms, however, constructing a proxy pathogenic truth set is more challenging, as true pathogenic DN variants are expected to be rarer than canonical LOF variants. Importantly, the most critical driver of OddsPath towards pathogenicity is the benign truth set size. OddsPath is particularly sensitive to having a sufficient number of variants correctly classified as normal. In practice, it is often possible to use the same benign truth set for both an LOF and a DN assay for the same gene. If variants in that benign set are correctly classified as normal by both assays, similar levels of OddsPath-derived evidence towards pathogenicity can, in principle, be reached for both mechanisms.

• ii)Choosing an appropriate method to generate an integrated functional score

In choosing a method for combining multiplexed functional data and generating an integrated functional score, a user must carefully consider their end goal. If interpretability of a model is an important consideration, statistical methods such as logistic regression, which allows quantification of the weighting of covariates in the combined model, is recommended. If performance is the primary concern, a more complex but less interpretable supervised model, such as a naive Bayes or a random forest classifier, may be used. If the goal is to explore patterns without labeled data, unsupervised methods such as k-means clustering or PCA are appropriate.

Beyond the primary goal, practical considerations may influence the method of choice. When two MAVEs measure different features of a protein - for instance, where one reports on protein stability while another reports on protein function, it is important to carefully consider the gene function and disease mechanism to assess if these two signals might be conflicting before choosing an integration methodology. For example, genomic assays permit the interrogation of variant impact on splicing, unlike cDNA-based assays. In such cases, supervised methods such as logistic regression, naive Bayes, or random forest automatically weigh features (in this case, different multiplexed functional datasets) based on their ability to classify clinical truth sets. On the other hand, methods such as PCA will need to be manually weighted based on individual MAVE performance metrics (e.g., sensitivity, specificity) or OddsPath values calculated on held-out validation data. However, the choice of weights must be carefully considered and justified to avoid introducing user bias.

An alternative approach is to remove specific variants from certain multiplexed functional datasets to ensure that conflicting signals are resolved before combining the data. For example, computational prediction tools (e.g., SpliceAI [74], Pangolin [75], MaxEntScan [76]) can be used to flag and exclude splice-affecting variants from cDNA-based assays prior to integration. However, any such exclusions must be documented transparently. A related consideration is whether an end user wants to only use complete cases (i.e. data is available for the variant in all MAVEs) or would like to use all data irrespective of some missing values. Methods like random forest classifiers handle missing values by imputing the median for each class during training. However, this approach may not be appropriate for clinical decision-making. We recommend using only complete cases (variants scored by all MAVEs) for clinical classification, while exploratory analyses may include all available data.

Finally, the size of the truth set will also influence which models are appropriate to use (Fig. 2B). For genes with small truth sets (10–100 variants), complex supervised learning methods, such as the random forest method, are not recommended, as such methods often require a training/test split, and too few variants will remain in the test set to adequately assess categorization. Instead, statistical methods or unsupervised learning methods are recommended in these cases. For genes with larger truth sets, complex supervised learning methods may be explored alongside statistical methods and unsupervised learning.

Supervised methods

Supervised approaches, including both statistical and machine learning methods, use labeled data to train a model to predict a specific outcome, in this case normal versus abnormal categorization of B/LB vs. P/LP truth set variants. These approaches can be powerful but require a sufficiently large truth set (which varies by method). The training set will be used to train the machine learning classifier, while the held-out test set will be used to evaluate the classifier’s accuracy on unseen data not used in training. K-fold or leave-one-out cross validation is recommended to avoid overfitting.

Logistic regression is a more readily interpretable supervised approach that can categorize variants as normal or abnormal in function. Logistic regression estimates a coefficient for each variable that indicates the importance of that feature or dataset to the final categorization. Logistic regression can perform well even for smaller datasets and with fewer computational resources. One key drawback of logistic regression is that it cannot handle missing data, which can be an issue when combining multiple MAVEs that assess non-overlapping sets of variants.

Machine learning approaches such as naive Bayes classifiers and random forests, while not readily interpretable, can perform very well on large datasets. Such approaches can be advantageous where there is a non-linear relationship between the features and the outcome, as well as interactions across multiplexed functional datasets. For example, random forest classifiers can handle complex relationships across datasets, whereas naive Bayes approaches assume that features are independent, which may not hold in some scenarios.

In one published example, a naive Bayesian classifier was used to classify variants by combining multiplexed functional data for the TP53 gene to generate a single functional prediction [55, 56, 60]. None of the constituent MAVEs perfectly discriminated between known pathogenic and benign variants, with the highest applicable evidence strength for any constituent assay being PS3_moderate. By contrast, the combined classifier approach achieved an OddsPath of 30.3 for functionally abnormal variants and 0.054 for functionally normal variants. This enabled application of strong evidence towards pathogenicity (PS3) and moderate evidence towards benignity (BS3) for the combined model. Combining predictions from this classifier with other available data enabled reclassification of 69% of TP53 VUS [60].

Unsupervised approaches

Unsupervised methods use unlabeled data to find patterns without external training data. When there is no truth set available (i.e. no labeled data to train on) or the size of the truth set is insufficient for supervised machine learning approaches, unsupervised approaches such as PCA and k-means clustering could still identify patterns in the data that might inform the consequence of a variant either in terms of function or in terms of pathogenicity. Unsupervised clustering algorithms such as t-SNE and UMAP, widely adopted for the analysis of single-cell RNA sequencing (scRNA-seq) [77], enable visualizations of cell-type clusters based on shared gene expression patterns [42]. When applied to multiplexed functional data, these algorithms are not skewed by external factors such as training datasets derived from ClinVar, which can be biased by ascertainment and ancestry [1, 78, 79]. Most unsupervised methods are relatively immune to overfitting and can perform well when truth sets are unavailable or limited. Another advantage of unsupervised methods is that they allow the full truth set to be used (i.e., without splitting into training and test sets) to validate the model and assess the strength of evidence for clinical variant classification.

Finally, unsupervised approaches like Gaussian mixture modeling are better at capturing complex or high-dimensional relationships between multiplexed functional datasets such as time course data or an assay that can read out variable degrees of function to identify subgroups of variant classes instead of a binary classification. For example, across multiple studies, Gaussian mixture modeling of multiplexed functional data collected over a time course has identified variants that appear to exert a milder effect in vitro (growing more slowly than other variants) than full loss-of-function variants [9, 10]. Radford et al. demonstrated separate populations of fast- and slow-depleting DDX3X variants, in which slow-depleting variants were hypothesized to exert milder effects on protein function [10]. Confirmation of whether such observations correlate with patient phenotypes requires large, well-phenotyped clinical cohorts, which are lacking for most rare-disease genes.

Model performance metrics

The performance of a specific model can be evaluated by comparing the metrics outlined in Fig. 1C, such as specificity and sensitivity, relative to the individual input MAVEs. Additionally, one must be cautious of the potential for overfitting when combining multiplexed functional data to maximize apparent performance in clinical variant discrimination. Exploring multiple combination strategies and selecting the one that performs best on the test set risks inflating apparent performance, which may not generalize to other variants. To alleviate this issue, we recommend evaluating variant classification via k-fold or leave-one-out cross-validation. While the above framework provides a starting point for a user to consider different approaches to combining MAVEs, users should adapt it based on each method’s assumptions, relative strengths and weaknesses, and the intended application of the results. In the end, no single approach will be suitable for every use case, and we recommend that each user be guided by their own domain expertise on gene function, the specific MAVEs being combined, and the gene-disease dyad (see the decision tree in Fig. 2B to guide method selection).

When combining multiplexed functional data, one may observe improvements in some performance metrics while others decline. In such cases, OddsPath as well as clinical context should guide the selection of the right model. For example, when a clinical diagnosis of a condition has already been made, prioritizing sensitivity (successfully identifying true pathogenic variants) over specificity (avoiding misclassifying truly benign variants as pathogenic) may be appropriate if pathogenic variant classification affects patient treatment or management options. Similarly, when PPV and NPV move in opposite directions, it becomes important to consider whether the clinical context calls for more pathogenic or more benign assignments. A key variable that influences all these metrics is the threshold value(s) set to call a variant abnormal or normal with respect to function (see Section iii for more guidance on threshold setting). Ultimately, OddsPath values can serve as tie-breaker criteria in these cases because they translate directly to clinically applicable evidence strength categories (PS3/BS3) and provide a standardized framework for evaluating assay performance in both pathogenic and benign directions.

• iii)Setting thresholds to separate functionally normal and abnormal variants

It is worth noting that threshold-setting is a necessary step for any MAVE-based functional evidence assignment, not only for integrated scores. In general, the goal for setting thresholds is to achieve high confidence and the strongest possible evidence strength for or against pathogenicity. While single-threshold methods are simpler to implement and allow categorization of all variants, they can lack nuance and have a higher risk of misclassification near the threshold. Double threshold methods that use separate thresholds for benign and pathogenic categorization require more user discretion and result in more “indeterminate” calls but can minimize misclassifications if implemented appropriately. Clinical context and the potential consequence of misclassification should be carefully considered in choosing the appropriate thresholding strategy.

In many cases, the distributions of scores for proxy truth set variants (such as nonsense and synonymous variants) are used to set thresholds. However, this approach can be problematic for the same reasons as using such variants as the truth set is problematic (described in Section ii). To avoid overreliance on proxy truth set variants, we recommend using the score distribution of clinically validated P/LP and B/LB variants wherever possible to select appropriate thresholds.

There are multiple schools of thought on how best to set thresholds for separating functionally normal from functionally abnormal variants. In one approach, percentiles are used to determine thresholds. In Boyle et al. [80], the 5th percentile of functional scores for synonymous CYP2C19 variants was used as a threshold to distinguish between “WT-like” and “decreased” scores. In other studies, statistical modeling of variant distributions has been used. For instance, Nagy et al. [81] modeled scores of missense variants in a control condition as a normal distribution and a threshold was selected based on the bottom first percentile of the distribution. Alternatively, receiver operating characteristic (ROC) curve analysis can be used to identify thresholds that optimize the balance between sensitivity and specificity based on known pathogenic and benign variant sets. While the examples given here are for individual multiplexed functional datasets, the same principles apply to an integrated dataset. Similar methods can be used to set thresholds for the integrated score that distinguish functionally normal from functionally abnormal variants. Regardless of the threshold-setting approach used, it is important to validate threshold performance using independent, clinically validated variant truth sets.

• iv)Evaluating the utility of the integrated evidence strength assignment

The performance of an integrated score or classification, relative to each individual MAVE, should be assessed to determine whether it has improved over the individual assays (Fig. 2). In addition to assessing classification performance with standard metrics such as sensitivity, specificity, PPV, and NPV, the main objective is to determine whether OddsPath analysis shows that integration provides greater evidence strength for pathogenicity, benignity, or both. This includes evaluating whether the difference in dynamic range between functionally normal and abnormal variants increases when using the integrated score relative to the best-performing individual MAVE. If the integrated score does not yield a clear improvement over the best-performing input MAVE, we recommend using the score for the best-performing individual MAVE to determine which functional score to translate into functional evidence for clinical variant classification.

It is important to emphasize that combining multiplexed functional data will not necessarily lead to an improved clinical utility, such as an increased number of variants reaching higher evidence strength, for every gene-disease dyad. In such cases, there may still be a use case for combining multiplexed functional data in a basic research context, as it may yield better understanding of protein structure-function relationships and/or identify unexpected variant clusters which yield novel insight into biology. For example, a gene with multiplexed functional data from multiple MAVEs spanning function, protein abundance, subcellular localization, and protein-protein interactions might reveal distinct functional classes of variants, such as loss-of-function versus gain-of-function or dominant-negative. Functional categorization information could then feedback to clinical practice through improved understanding of disease mechanism.

Illustrative examples (TP53, LDLR, and PTEN)

To illustrate some of the concepts above, we present three brief examples of combining multiplexed functional data with different methodologies for the genes TP53, LDLR, and PTEN. We generated these examples using the Shiny application we created for the purpose of easily evaluating methods to combine such data and evaluate the utility of an integrated score (Fig. 3). For these analyses, we focused on determining whether integration enhanced clinical utility by achieving higher evidence strength categories (as determined by OddsPath calculations), restricting analysis to variants scored across all contributing assays.

Fig. 3.

Example of combining multiplexed functional data for TP53 MAVEs. A The number of variants of different classes present in the constituent assays is indicated; P/LP and B/LP variants correspond to the clinical truth set; PTC and Syn variants constitute the proxy truth set. “Clin/Proxy Ratio” refers to the fraction of clinical truth set combined with proxy truth set for the purpose of model training (e.g., 0.7/0.01 = 70% of clinical variants, 1% of proxy variants). “Train/Test Split” refers to the percentage of clinical missense variants used in model training versus held out for testing. B Principal component analysis of the scores in five constituent TP53 datasets. The first two principal components are shown. Variants were clustered by class within the dataset for easy visualization. C Example k-means clustering output plotted to visualize candidate clusters (n = 4) within principal component space. D Comparison of sensitivity, specificity, PPV, and NPV for individual TP53 MAVEs with integrated scores using different methodologies. E For each method, we calculated the OddsPath for variants classified as functionally abnormal (top) or normal (bottom) to determine what strength of evidence could be applied towards pathogenicity or benignity, respectively, compared to the best-performing single MAVE

For TP53, we used five MAVEs from three publications [55–57]. These datasets were deliberately selected for combination as they span multiple different cell types (A549, MOLM13, and HCT116) and utilize both positive and negative selection using compounds nutlin-3 or etoposide, depending on the expression of functional TP53 [55, 56]. Further, three of these experiments were designed to screen for LOF variants while another two were designed for DN variants. Pathogenic variants in TP53 that cause Li-Fraumeni syndrome are known to act through both LOF and DN mechanisms. We analyzed these datasets with a clinical truth set of 281 missense variants (223 B/LB and 58 P/LP) carefully selected to avoid classifications that relied on functional data (see Methods above for detailed description). From this truth set, 201 variants (143 B/LB and 58 P/LP) were scored in all five TP53 MAVEs (Fig. 3A). As this represents a well-balanced and appropriately sized truth set, we used 70% of these variants to train the model, with negligible inclusion of proxy truth set variants (1% of synonymous and PTC variants) in the training data.

To explore these data, we plotted missense B/LB and P/LP variants in principal component space based on their scores across the five individual MAVEs, as well as in k-means clusters (Fig. 3B-C). These data form distinct clusters, suggesting underlying patterns that distinguish B/LB and P/LP variants. We then employed both unsupervised and supervised learning methods to generate a single score representing the integration of all five multiplexed functional datasets and evaluated both individual and integrated scores to assess performance (Fig. 3D; see Additional File 1: Tables S1-S4 for classifier outputs, threshold values, and comprehensive model metrics). Individual MAVEs showed variable performance, with sensitivity ranging from 0.897 to 0.931 and specificity ranging from 0.629 to 0.944. The DN_reporter MAVE [56] performed best overall, achieving the highest specificity (0.944) and PPV (0.869) (Fig. 3D). A classifier based on PC1 from Principal Component Analysis did not improve specificity compared to the best-performing individual MAVE, though it did slightly improve sensitivity. The naive Bayes classifier improved specificity and achieved BS3_strong evidence strength for OddsPath normal (Fig. 3E), with a 3-fold cross-validation kappa value of 0.752. The random forest classifier, on the other hand, improved specificity and sensitivity, as well as led to significant improvement in OddsPath for both functionally abnormal and normal variants, achieving PS3_strong evidence strength (Fig. 3E; 3-fold cross-validation z-score = 0.098). Note that both supervised methods do have some degree of overfitting to training data in identifying benign variants as functionally normal, as shown by the specificity value of 1 vs. ~0.9 in the train/test split. These results suggest that while integration may not improve basic classification metrics, it can enhance clinical utility by providing stronger evidence assignments, though careful validation and potential model refinement are needed to address overfitting.

While LDLR has two independent MAVE publications available [40, 41], for this worked example, we combined the two score sets from Tabet et al., measuring cell-surface abundance and LDL uptake [40]. We analyzed these datasets with a clinical truth set of 371 missense variants (15 B/LB and 356 P/LP) (Fig. 4A). As this is a relatively unbalanced truth set, we included 60% of the available proxy truth set variants to aid model training. 40% of missense variants were used for model training, while the remaining 60% of variants were reserved as a test set.

Fig. 4.

Example of combining multiplexed functional data for LDLR and PTEN MAVEs. A The number of variants of different classes present in the constituent assays is indicated; P/LP and B/LB variants correspond to the clinical truth set; PTC and Syn variants constitute the proxy truth set. “Clin/Proxy Ratio” indicates the proportion of clinical truth set/proxy truth set used in model training (e.g., 0.5/0.6 = 50% of available clinical variants, 60% of available proxy variants). “Train/Test Split” refers to the percentage of clinical missense variants used in model training versus held out for testing. B Principal component analysis of the scores in two constituent LDLR datasets. The first two principal components are shown. Variants were clustered by class within the dataset for easy visualization. C Example k-means clustering output plotted to visualize candidate clusters (n = 4) within principal component space. D Comparison of sensitivity, specificity, PPV, and NPV for individual TP53 MAVEs with integrated scores using different methodologies. E For each method, we calculated the OddsPath for variants classified as functionally abnormal (top) or normal (bottom) to determine what strength of evidence could be applied towards pathogenicity or benignity, respectively, compared to the best-performing single MAVE. (F-J) Same plots as in (A-E) but for PTEN

Plotting the missense B/LB and P/LP variants in principal component space based on their scores in the two individual MAVEs, as well as based on k-means clusters (Fig. 4B-C), reveals distinct clusters that distinguish B/LB and P/LP variants. It is illustrative to follow a single variant and how it is partitioned by the PCA-based approach. LDLR p.D221G is a known pathogenic missense variant that meets the criteria for normal abundance by the abundance MAVE and loss of activity by the activity MAVE. Extracting the first principal component for this variant gives a value of -0.226. The average PC1 value for all tested variants with normal abundance and loss of activity is -0.26 (n = 4,316). In contrast, the average PC1 value for all tested variants with normal abundance and normal activity is -0.93 (n = 5,071). For comparison, the average PC1 value for all tested variants with abnormal, reduced abundance is 0.87. In this example, negative PC1 values are associated with normal abundance, and the relative negative value separates variants with normal activity from those with loss of activity.

Individual MAVEs showed variable performance, with sensitivity ranging from 0.604 to 0.787 and specificity ranging from 0.933 to 1. The activity MAVE [40] performed best overall, achieving the highest specificity (1.00) and sensitivity (0.787) (Fig. 4D; See Additional File 1: Tables S5-S8 for classifier outputs, threshold values, and comprehensive model metrics). A classifier based on PC1 of the principal component analysis slightly improved sensitivity compared to the best-performing individual MAVE due to three fewer false negative variants. The naive Bayes classifier performed similarly to PC1, with similar sensitivity and specificity between training and test splits (Fig. 4E). The random forest classifier, on the other hand, improved sensitivity without sacrificing specificity across the full truth set and each individual training and test split.

For the full truth set, the random forest improved OddsPath for both functionally abnormal and normal variants (Fig. 4E), although neither score crossed the threshold for “strong” evidence. It is important to note that there is some discrepancy in performance between the random forest on the training and test splits, suggesting mild overfitting (Fig. 4D). We posit that the results presented herein support the potential of supervised learning to integrate LDLR MAVEs. However, further work would be needed to optimize the random forest model to minimize overfitting.

For PTEN, we used two datasets that measured protein abundance and fitness effects of different variants [49, 50, 61]. A clinical truth set of 38 B/LB variants and 216 P/LP variants was retrieved from ClinVar (accessed on 11 June 2025), with the caveat that functional data may have contributed to variant classification for some of these variants, potentially introducing bias and circularity to this analysis. The truth set variants were merged with the PTEN multiplexed functional datasets to obtain variants scored in both datasets (16 B/LB and 105 P/LP variants; Fig. 4F). As this truth set is heavily biased toward P/LP variants, we included all synonymous (presumed benign) and nonsense (presumed pathogenic) variants in the MAVE as part of our proxy truth set, along with 40% of the clinical truth set (Fig. 4F). Note that this proxy truth set is also biased towards presumed pathogenic variants (319 PTC; 87 Syn).

Principal component analysis and k-means clustering show very little overlap between presumed pathogenic variants (PTCs) and clinically validated P/LP variants, suggesting that clinical P/LP variants exhibit more nuanced functional differences than complete loss-of-function (i.e., PTC variants) (Fig. 4G-H). In contrast, presumed benign variants (synonymous) cluster closely with clinically validated B/LB variants in PC space, which may be due to the small number of variants in both classes, as well as the potential use of functional data in ascertaining the benignity of the B/LB variants, i.e., circularity (Fig. 4G).

Individual assays as well as the integrated models have relatively modest sensitivity but very high specificity, i.e. the pathogenic/abnormal calls are more reliable than the benign/normal calls (Fig. 4I; see Additional File 1: Tables S9-S12 for classifier outputs, threshold values, and comprehensive model metrics). This is also likely due to the small number of B/LB variants in the clinical truth set for PTEN, as well as the relatively skewed distribution of nonsense and synonymous variants among proxy truth set variants. PCA modestly improved sensitivity (0.819) while maintaining perfect specificity. Naive Bayes achieved reasonable training performance (sensitivity = 0.850, NPV = 0.571) that was maintained on test data (sensitivity = 0.892, NPV = 0.533). The random forest model exhibited some evidence of overfitting: sensitivity degraded from 1.000 to 0.908 and NPV from 1.000 to 0.571 in the test data compared to the training data. Moreover, the integrated scores do not provide a higher PS3 strength of evidence for pathogenicity or benignity compared to individual MAVEs and therefore do not provide increased clinical utility (Fig. 4J). The fundamental limitation here is likely the inadequate number of clinically validated B/LB variants, which prevents robust model training and calibration. Based on these results, we recommend revisiting PTEN data integration when additional clinically validated B/LB variants become available. More broadly, this example illustrates a general limitation of using PTCs as a proxy for pathogenic truth sets: it assumes that PTCs share functional properties with typical P/LP missense variants, which may not always hold. As seen here, PTC variants can exhibit more severe functional deficits than the spectrum of clinical P/LP variants, which can affect model training and OddsPath calibration.

In summary, by evaluating various model metrics and the final OddsPath values for pathogenicity and benignity, we provide a framework for assessing whether an integrated functional score generated by a specific model is reliable and has practical utility. Our examples show that integration can yield moderate (TP53) to mild (LDLR and PTEN) benefits, depending on the specific gene and the availability of robust truth sets for model training. We anticipate that the Shiny app we have developed will provide easy access to a set of standard integration methods, lowering the barrier to similar exploratory analyses for end users.

Alternative approaches in the literature

Here, we have taken the approach where the input is a data matrix containing functional scores from independent assays, and the output is a single integrated score for each variant. This approach allows the calculation of an OddsPath to assign PS3/BS3 evidence codes to all variants at a specified evidence strength. It is worth acknowledging that there are other frameworks for integrating multiple lines of functional evidence in variant classification, including gene-specific approaches and decision tree models developed for specific genes such as DNA mismatch repair genes, TP53, and BRCA1 [82–85]. Moreover, the OddsPath framework does have certain caveats. The OddsPath value represents the likelihood ratio for all variants above or below a set threshold for functionality, rather than for a specific variant at a particular score. Therefore, it is prone to overestimating confidence near the threshold and underestimating confidence far from the threshold. An example of an alternative approach is the log-likelihood ratio method used by van Loggerenberg et al. [86], which may be particularly useful when variant scores exhibit a wide dynamic range. The log-likelihood ratio approach is more granular; each variant is ascribed a PS3/BS3 evidence code at a different evidence strength. Using this approach on the HMBS gene allowed assignment of PS3/BS3 ACMG/AMP codes at either moderate or supporting evidence strength based on thresholds derived from the log-likelihood ratios [86].

Discussion

Limitations and future directions for integrating multiplexed functional data

The methodologies we have applied to explore integrating multiplexed functional datasets and the Shiny application developed for this purpose could be improved in several ways: (1) Support for analyzing variants that are only sampled by a subset of assays needs to be enabled for methods that permit missing values, such as a custom implementation of naive Bayes classifier that ignores missing attributes; (2) The ability to integrate normal/abnormal calls from low-throughput “gold standard” functional assays used by ClinGen VCEPs and any assays that produce categorical labels rather than continuously distributed scores; (3) More nuanced models designed for combining specific types of data, such as time course data [87, 88] or imaging data; (4) Implementing a framework for quantitative integration of standard error metrics from functional assays into variant categorization evidence strength. By making the code for the Shiny application available on GitHub (an open-source platform for collaborating on software development), we hope that community-driven updates will add functionality to this tool over time.

Dissemination of integrated multiplexed functional data and unification of clinical application

Clinical application of an integrated score representative of multiple MAVEs requires the deposition of results into the public domain such as MaveDB [89]. It is imperative that the integrated functional scores be clearly signposted as being derived from a MAVE combination to avoid double-counting of data from both combined datasets and their constituent studies (a functionality built into MaveDB [89]), and labelled with respect to the relevant gene-disease dyad for clinical use to guard against inappropriate application. Similarly, combination methodologies that incorporate additional evidence sources, such as in silico variant effect predictions or allele frequencies, must be clearly highlighted to ensure evidence codes relating to contributing evidence types are not applied alongside functional evidence. Where possible, the evidence strengths that can be applied towards pathogenicity (PS3) and benignity (BS3) for the combined datasets under the Brnich et al. framework [20] should also be highlighted to steer appropriate clinical application. Consistency in the integration of multiplexed functional data would further benefit from expert review and incorporation into clinical guidelines, such as by a relevant ClinGen VCEP. This may include iterative evaluation of MAVE performance against up-to-date truth sets as the number of classified variants available to comprise these truth sets increases.

Improved classification of reduced penetrance variants and variable expressivity

Most MAVEs produce a quantitative measurement of variant effect. However, the OddsPath framework converts this to a categorical evidence strength for use in clinical variant classification. The extent to which MAVE scores reflect quantitative differences in variant function, and whether this is relevant for human phenotypes in health and disease is currently being explored [22, 90, 91]. Classification of variants falling in the intermediate range of MAVE scoring remains particularly challenging owing to difficulty in distinguishing truly intermediate readout from technical noise. Combined MAVE approaches offer a powerful way to address the effects of such variants: combining may mitigate the effects of experimental noise and allow assignment of evidence toward pathogenicity or benignity under a fully penetrant model. Alternatively, consistent observations of intermediate variant effect between assays or within a single assay (i.e. intermediate slope in a time course assay) may indicate the incomplete penetrance and/or variable expressivity of that variant [10, 12]. As clinical guidance on the classification and reporting of such variants continues to develop [92], integrating multiplexed functional datasets may improve confidence in assigning hypomorphic functional impact.

Insights into the genetic basis of disease

Outside the clinical context, MAVE combination may yield further insights into biological mechanisms of disease: as discussed above, unsupervised learning approaches may allow clustering of variants with shared characteristics that may not be apparent when considered at the individual assay level. Variant clusters that do not reflect known relationships between variant function across different assay models may enable the elucidation of novel biological mechanisms [93]. When MAVEs, underpinned by the same assay methodology but using different biological models (e.g., different cell types), are combined, investigation of specific variant clusters may provide insight into the underlying biological differences between the models and generate hypotheses for future research and possible refinement of clinical interpretation. Similarly, appropriate integration of multiplexed functional data generated in models representing different time points (e.g., stages of development) [12] may allow a better understanding of the temporal dynamics of variant effect [10], which may be particularly insightful in the investigation of many developmental disorders, which take place against a rapidly changing genomic and transcriptomic program [9, 10, 12].

Conclusions

The rapid increase in the number of variants of uncertain significance (VUS) classifications in recent years has highlighted the urgent need for efficient, large-scale methods to resolve variant classification. MAVEs have emerged as powerful tools to address this challenge, allowing for the functional assessment of hundreds to thousands of variants in a single experiment. As the number of published MAVEs for individual genes continues to grow, the potential to combine data from multiple assays presents a promising avenue for improving variant classification accuracy. Moreover, the forthcoming v4.0 update to the ACMG/AMP guidelines is expected to be based on a more continuous evidence point-based system [94], with the existing boundaries between categorical evidence strengths being replaced by integer LLR value thresholds. Any improvements in applicable evidence through a combination of multiplex data, however modest, are more likely to cross these integer thresholds and improve applicable evidence strength than to cross the existing categorical thresholds and their LLR equivalents. By providing a framework for when and how to combine multiplexed functional data and an easy-to-use web tool to test different methods of integrating multiplexed data, we aim to accelerate progress in this field and ultimately improve patient care through more accurate and efficient genetic diagnoses.

Abbreviations

B/LB

Benign or likely benign

CDS

Coding sequence

DN

Dominant negative

GOF

Gain-of-function

Gene-Disease Dyad

Specific pairing of a gene to a specific clinical phenotype

Indel

Insertion or deletion

LOF

loss-of-function

MAVE

Multiplexed assay of variant effect

Molecular Phenotype

Phenotypic outcome measured by the MAVE

NPV

Negative predictive value

OddsPath

Odds of pathogenicity

P/LP

pathogenic or likely pathogenic

PPV

Positive predictive value

PTC

Premature termination codon

SGE

Saturation genome editing

SNV

Single nucleotide variant

VAMP-seq

Variant abundance by massively parallel sequencing

VUS

Variant of uncertain significance

Authors’ contributions

MD, SF, ER, JC, and SJ conceptualized the initial study. AM and MT provided the TP53 truth set with minimal circularity. JC wrote the Shiny web application and performed the MAVE dataset integration analyses presented herein. CR, JC, and SJ were major contributors in writing the manuscript. AS and LS provided significant feedback in revising the manuscript. All authors read and approved the final manuscript.

Funding

S.J. was supported by NIH R35 GM133433. L.M.S. was supported by R01HG013025 and UM1HG011969. A.B.S. was supported by an NHMRC Investigator Fellowship (APP177524). C.T. and C.F.R. are supported by CRUK Programme Award CG-MAVE (EDDPGM Nov22/100004). S.F. was supported by UM1HG011969 and RM1HG010461. M.D. was supported by U01HG011758, UM1HG011969, RM1HG010461, OT2OD002751, and CPRITRP210027. A.E.M. was supported R01HG013025, an Early Career Award from the Alex’s Lemonade Stand for Childhood Cancer and RUNX1 foundation (21-25037), and the Brotman Baty Institute Catalytic Collaborations Grant (CC28).

Data availability

The web Shiny application is available at https://jagannathan-lab.shinyapps.io/AVE_shinyApp_Jun2025/. The GitHub repository with Shiny application code, example csv input files, and detailed instructions is available at https://github.com/jagannathan-lab/2025-calhoun_et_al [95]. An archive of the repository is also available at Zenodo [96]. The TP53 MAVE datasets used herein are available at MAVEdb [97–101].The PTEN MAVE datasets used herein are available at MAVEdb [102, 103].The LDLR MAVE datasets used herein are available at MAVEdb [104, 105].

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.