Deep Learning and Experimental Validation of BRCA1-Associated Protein-1 Missense Variant Pathogenicity in Uveal Melanoma

Abstract

Objective

To benchmark the pathogenicity predictions of AlphaMissense, a deep learning model, against high-throughput functional scores from saturation genome editing (SGE) for all experimentally tested BRCA1-associated protein-1 (BAP1) missense variants in uveal melanoma (UM).

Design

Cross-sectional analytical study comparing computational predictions with experimentally derived functional classifications and clinical annotations.

Subjects, Participants, and/or Controls

ClusteredAll 4619 BAP1 single amino acid substitutions profiled in a published Regularly Interspaced Short Palindromic Repeats–Cas9 SGE viability assay. No separate control cohort was required. Clinical Variant Database (ClinVar)–annotated variants within this set served as an independent reference.

Methods

Saturation genome editing log₂-fitness scores were dichotomized at a validated depletion threshold. Prediction tools AlphaMissense, Rare Exome Variant Ensemble Learner (REVEL), Meta-predictor based on a Recurrent Neural Network (MetaRNN), and Combined Annotation Dependent Depletion (CADD) scores were aligned to SGE classifications. Diagnostic performance was assessed using receiver operating characteristic (ROC) and precision–recall (PR) analyses. Gene-specific threshold optimization for AlphaMissense was performed using Youden index. Structural mapping was performed using the AlphaFold2 BAP1 model.

Main Outcome Measures

(1) Concordance between in silico predictors and SGE labels (area under ROC and PR curves); (2) agreement with ClinVar classifications; (3) structural clustering of high-risk residues.

Results

Of the 4619 BAP1 missense variants tested by SGE, 988 (21.4%) were experimentally classified as pathogenic. Using a default genome-wide threshold of 0.564, AlphaMissense predicted 2563/4618 variants (55.5%) as pathogenic. Receiver operating characteristic analysis showed good overall performance, with an area under the curve of 0.837, indicating that AlphaMissense pathogenic calls were the most strongly associated with experimentally defined pathogenicity when compared to REVEL, MetaRNN, and CADD. Receiver operating characteristic analysis informed a BAP1-specific optimized threshold of 0.952, improving sensitivity–specificity balance and identifying 1556/4618 variants (33.7%) as pathogenic. Of the 13 851 possible BAP1 missense variants, AlphaMissense predicted 8694 and 5816 as pathogenic by the genome-wide and optimized thresholds, respectively. Structural mapping highlighted clusters of high-risk substitutions within the catalytic core and protein-interaction motifs of the BAP1 protein. Finally, AlphaMissense predictions matched 11 of 12 ClinVar-classified variants (91.7%).

Conclusions

AlphaMissense aligns closely with the gold standard functional data for BAP1, providing a rapid, scalable, and interpretable approach to variant classification in UM. This study introduces an optimized, gene-specific threshold that further enhances its precision, supporting the integration of artificial intelligence–based tools into ocular oncology precision-medicine pipelines for risk stratification and clinical decision-making.

Financial Disclosure(s)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Uveal melanoma (UM) is the most common primary intraocular malignancy in adults and arises from melanocytes within the iris, ciliary body, or choroid.^1,2 Despite advances in early diagnosis and local control of primary tumor, approximately 40% to 50% of patients ultimately develop systemic metastases, most commonly to the liver, which carries a median survival of less than 1 year despite treatment.2, 3, 4 Long-term survival analyses confirm that about 2 in 5 patients die from UM within 20 years after primary tumor treatment.⁵ The clinical trajectory of UM is primarily dictated by underlying driver and promoter genetic alterations, with inactivation of the tumor suppressor gene BRCA1-associated protein-1 (BAP1) serving as a sentinel event.⁶ The BAP1 gene encodes for the BAP1 protein, which functions as a deubiquitinating enzyme, playing a key role in chromatin remodeling, DNA repair, and cell cycle regulation.⁷ Mutations in BAP1 result in the expression of an altered BAP1 protein that can result in a dedifferentiated stem-like state that is strongly associated with adverse clinical features, including early metastasis, and reduced overall survival, thereby classifying such tumors as high risk.^4,6,7, 8, 9, 10

Given the critical role of genetic alterations on driving the clinical progression of UM, the current prognostication landscape has been informed by advances in genetic medicine. Gene expression profiling stratifies tumors into class 1 (low metastatic risk) and class 2 (high metastatic risk), with BAP1 loss serving as a key marker of increased metastatic risk.^7,9,11,12 Preferentially Expressed Antigen in Melanoma status, introduced by the Collaborative Ocular Oncology Group, further refines gene expression profiling prognostication, as Preferentially Expressed Antigen in Melanoma positive tumors across both classes is associated with worse prognosis and increased metastatic potential.¹⁰ Complementing gene expression profiling, The Cancer Genome Atlas, developed by the National Cancer Institute and the National Human Genome Research Institute, classifies UM into 4 genomic subtypes defined by chromosomal status and somatic mutations.^13,14 These subtypes, A through D, correspond to progressively worse prognosis.¹⁵ Type A tumors are characterized by disomy of chromosomes 3 and 8, and often EIF1AX mutations.^16,17 Type B tumors retain disomy of chromosome 3, gain 8q, and frequently contain SF3B1 mutations.16, 17, 18 Type C tumors exhibit monosomy 3 or 8q gain and often present with BAP1 inactivation.^13,16 Type D tumors contain monosomy 3 with multiple 8q gains and are associated with an inflammatory phenotype.^7,13,16 Together, these frameworks highlight the importance of mutation-driven classification in refining prognostication in UM, with BAP1, which is located on chromosome 3, playing a recurring role as a determinant of metastatic risk.

Inactivation of the BAP1 protein not only drives metastasis but also underlies the autosomal dominant germline BAP1 tumor predisposition syndrome which confers susceptibility to UM, renal cell carcinoma, mesothelioma, cutaneous melanoma, and other malignancies.^7,19,20 Given its pivotal role in tumor biology and patient prognosis, accurate interpretation of BAP1 gene variants is vital for risk stratification, surveillance planning, and patient specific clinical decision-making.⁹ However, classifying BAP1 gene variants remains a persistent challenge. Many missense alterations are labeled as variants of uncertain significance (VUS) in publicly available clinical genetic databases such as the Clinical Variant Database (ClinVar), limiting their utility in precision oncology.^9,21,22

Recent advances in both experimental and computational genomics provide complementary strategies for determining the significance of BAP1 gene variants. Saturation genome editing (SGE), a Clustered Regularly Interspaced Short Palindromic Repeats–based approach, enables high-throughput, functional testing of nearly all possible single-nucleotide variants (SNVs) within a gene. In a landmark study, Waters et al²³ utilized SGE to functionally characterize 4619 BAP1 missense variants, generating a comprehensive map of their pathogenic potential. These functional scores serve as an experimental gold standard, capturing the biological impact of each mutation in a cellular model system.

Recently, artificial intelligence (AI)–driven tools, such as AlphaMissense, have emerged as powerful predictors of variant pathogenicity.²⁴ Derived from the protein structure predictor AlphaFold, AlphaMissense is a deep learning model developed by Google DeepMind to assess the pathogenicity of missense variants.^24,25 AlphaMissense integrates sequence conservation, predicted three-dimensional structure, and population-level variant frequency to assign a pathogenicity score to each amino acid substitution, ranging from 0 to 1.²⁴ The model categorizes variants as Likely Pathogenic, Uncertain, or Likely Benign based on its prediction. While AlphaMissense has demonstrated strong performance across the human proteome, its accuracy in the context of UM, particularly for the BAP1 protein, has not been extensively benchmarked against experimentally derived functional data largely due to a paucity of available data.²²

In addition to AI predictors and experimental functional data sets, clinically curated germline variant cohorts provide a factual framework for variant interpretation. Walpole et al²⁶ performed a comprehensive analysis of germline BAP1 variant carrying families worldwide to classify missense variants. Such clinically curated data sets offer an opportunity to directly compare computational predictions, experimental results, and clinical interpretations.

Building on our team's prior work, which evaluated AlphaMissense predictions against ClinVar annotations across key UM-associated genes including BAP1, we identified critical gaps limiting its clinical translation.²² Specifically, there is a lack of detailed characterization for BAP1-specific missense variants in the literature and the absence of direct functional benchmarks. This unmet need underscores the importance of scalable strategies, such as large-scale functional assays and AI-based pathogenicity predictors that can refine variant classification and accelerate the integration of genetics into personalized management of UM.27, 28, 29 The present study addresses these gaps by focusing exclusively on BAP1 gene missense variants and benchmarking AlphaMissense predictions against other AI predictors and experimentally derived SGE functional scores. Through this targeted approach, we aim to refine pathogenicity thresholds specific to the BAP1 gene and bridge the translational gap between computational predictions, experimental validation, and clinical decision-making, possibly informing future precision medicine efforts in UM.

Methods

BAP1 Protein and Sequence

The BAP1 protein is a 729 amino acid deubiquitinating enzyme (UniProt ID: Q92560) with established tumor suppressor function. The full-length canonical protein sequence was retrieved from UniProt for downstream computational analysis (https://www.uniprot.org/uniprotkb/Q92560/entry#sequences, accessed on 10/25/2024).³⁰ Considering all possible single amino acid substitutions across the protein, a total of 13 851 unique missense variants (729 positions × 19 alternative amino acids) were included in the evaluation. This comprehensive set of possible variants served as the basis for AlphaMissense pathogenicity scoring and structural mapping.

Saturation Genome Editing Data Set

AlphaMissense pathogenicity scoring was benchmarked against a recently published SGE study of BAP1 by Waters et al,²³ which functionally profiled 4619 unique missense variants using a Clustered Regularly Interspaced Short Palindromic Repeats–Cas9–mediated in vitro cell viability assay. In their study, thousands of BAP1 SNVs were introduced into HAP1 cells, a near-haploid human cell line that readily reveals the direct effects mutations since only a single allele is present. For each edited cell population, growth was tracked over time and summarized as a log₂-transformed functional score representing relative cell fitness. Variant cell populations below a score of –0.038 were classified as depleted (pathogenic), while populations above this threshold were classified as unchanged or enriched (nonpathogenic). This cutoff was validated against ClinVar, a database of clinically interpreted variants of genes including BAP1 and achieved >99% sensitivity and >98% specificity (area under the curve [AUC] = 0.998). Waters et al applied a Benjamini–Hochberg false discovery rate correction to reduce false-positives from testing thousands of variants in parallel.²³ Only variant cell populations with false discovery rate <0.01 were considered as being significantly different from wild-type and were subsequently categorized as depleted (pathogenic), unchanged (benign), or enriched (benign) based on their functional score.

Waters et al²³ tested a total of 18 108 variants of BAP1, of which 4619 were missense substitutions. This set represents approximately 33.3% of all 13 851 possible BAP1 missense substitutions, indicating that most theoretical variants have yet to be experimentally tested. The missense classifications from this data set (depleted, unchanged, or enriched) were obtained from the authors' public release (https://github.com/team113sanger/Waters_BAP1_SGE, accessed on 10/25/2024) and served as the high-confidence ground truth for evaluating AlphaMissense predictions.³¹

Clinical Variant Database Annotations

Clinical Variant Database is a publicly accessible archive of human genetic variants maintained by the National Institutes of Health and provides designations of pathogenicity, benignity, or uncertain significance.²¹ Clinical Variant Database contains expert curated annotations of variant-pathogenicity relationships derived from clinical testing results, research studies, and population data.²¹ Despite inherent variability in annotation quality, ClinVar remains one of the most comprehensive resources available on human genetic variants. For this study, ClinVar annotations accessed through the COSMIC (Catalogue of Somatic Mutations in Cancer) database (https://www.ncbi.nlm.nih.gov/clinvar/, accessed on 10/25/2024) were used to validate the clinical relevance of BAP1 missense variants and to benchmark AlphaMissense-derived pathogenicity scores.

AlphaMissense Pathogenicity Scoring

AlphaMissense is a deep learning model developed to predict the functional impact of amino acid substitutions across the human proteome.²⁴ AlphaMissense assigns each missense variant a continuous pathogenicity score ranging from 0 (benign) to 1 (pathogenic). To convert these continuous scores into categories, threshold cutoffs were applied. Variants with scores ≥0.564 were classified as Likely Pathogenic, while those <0.34 were classified as Likely Benign. Intermediate scores (0.34–0.564) fell into an Ambiguous category. This classification scheme, first proposed by Cheng et al,²⁴ achieved approximately 90% precision at the ≥0.564 pathogenicity score threshold when validated against genome-wide ClinVar annotations. AlphaMissense's categorical outputs, “Likely Pathogenic,” “Uncertain,” and “Likely Benign,” were also evaluated to assess model alignment with experimentally derived classifications.

Comparison with In Silico Predictors

To benchmark the performance of AlphaMissense against other in silico pathogenicity predictors, BAP1 missense variants characterized by SGE were aligned with prediction scores using a uniform variant annotation pipeline. All analyses were performed using the GRCh38 reference genome and the canonical BAP1 transcript (Ensembl ENST00000460680.6; UniProt Q92560).

Missense SNVs were annotated using Ensembl Variant Effect Predictor (offline mode) with the database of Non-Synonymous Functional Predictions plugin (v5.3.1a, GRCh38) to retrieve Rare Exome Variant Ensemble Learner (REVEL), Meta-predictor based on a Recurrent Neural Network (MetaRNN), and Combined Annotation Dependent Depletion (CADD) (PHRED-scaled) scores.32, 33, 34, 35, 36, 37 Rare Exome Variant Ensemble Learner is an ensemble-based meta-predictor that integrates multiple individual variant effect tools to generate a pathogenicity score (0–1).³⁴ Meta-predictor based on a Recurrent Neural Network is a deep recurrent neural network–based predictor trained on rare missense variants and similarly scaled from 0 to 1.³⁵ Combined Annotation Dependent Depletion integrates diverse genomic annotations into a PHRED-scaled score reflecting relative deleteriousness across the human genome.^36,37 Variants were integrated in 2 steps: AlphaMissense scores were merged with database of Non-Synonymous Functional Predictions predictors using normalized protein-level Human Genome Variation Society annotations for the canonical BAP1 transcript, and the combined data set was then aligned with SGE data using genomic coordinates (chromosome, position, alternate allele) under GRCh38. Pathogenicity thresholds were derived from literature recommendations (REVEL ≥ 0.5; MetaRNN ≥ 0.5; and CADD PHRED ≥ 20).34, 35, 36, 37

Given that the database of Non-Synonymous Functional Predictions–based predictors are defined at the nucleotide level, benchmarking analyses were restricted to SGE-profiled missense SNVs (n = 4723). These SNVs corresponded to 4263 unique protein substitutions after collapsing alternative codon encodings, representing 92.3% of the 4619 experimentally tested BAP1 missense changes. Variants lacking SGE functional scores were excluded from benchmarking analysis.

Concordance Analysis of Clinically Curated Germline BAP1 Variants

To further assess alignment between predictor scores, SGE functional data, and clinical classifications, we evaluated missense variants reported in the Walpole et al²⁶ germline BAP1 cohort. Missense variants reported by Walpole et al²⁶ were merged with the benchmarking data set at the protein level using a standardized 1-letter Human Genome Variation Society protein format. Duplicate variants reported across multiple families were collapsed at the protein level, retaining the highest American College of Medical Genetics and Genomics pathogenicity designation. For protein substitutions represented by multiple SNVs, the row corresponding to the most deleterious SGE functional score was retained, and associated predictor scores were used for analysis. Of the 41 variants reported by Walpole et al,²⁶ 34 were successfully matched and compared within our benchmarking data set.²⁶

Structural Visualization

To assess patterns of mutational intolerance in a structural context, AlphaMissense scores were mapped onto the AlphaFold2-predicted structure of the canonical BAP1 protein (https://alphafold.ebi.ac.uk/entry/Q92560?activeTab=annotations, accessed on 10/25/2024).²⁵ AlphaFold is a deep learning model developed by DeepMind that accurately predicts protein three-dimensional structures from amino acid sequences; its creators were awarded the 2024 Nobel Prize in Chemistry for this breakthrough.²⁵ Per-residue average pathogenicity scores were calculated across all possible amino acid substitutions at each position. The resulting structural model is color-coded on a blue-to-red spectrum, with blue representing low predicted pathogenicity (score ≈ 0), white representing ambiguous (score ≈ 0.5), and red indicating high predicted pathogenicity (score ≈ 1).³⁸ This visualization allowed us to identify intolerant protein regions, particularly within the catalytic core and protein interaction domains.

Statistical Analysis

Receiver operating characteristic (ROC) and precision–recall (PR) curve analyses quantified agreement between predictor scores and SGE-derived labels. Receiver operating characteristic AUC and area under the PR curve (AUPRC) were computed to assess discriminatory performance. Threshold optimization was performed using Youden index (maximizing sensitivity + specificity – 1), which identifies the cutoff that best balances sensitivity and specificity. F1-optimization (harmonic mean of precision and recall) was also applied, as it maximizes the trade-off between precision and recall and is particularly useful in imbalanced data sets. Kernel density estimation plots were used to visualize score distributions, providing an intuitive, smoothed view of how prediction scores differ between classes. Bootstrap resampling (2000 replicates) generated 95% confidence intervals (CIs) for AUPRC. Statistical analyses were conducted using Python (version 3.10).

Ethical Considerations

This study adhered to the ethical principles outlined in the Declaration of Helsinki. As the data analyzed were publicly available and fully deidentified, institutional review board or ethics committee approval from the University of Tennessee Health Science Center was not required. Additionally, because no patient-identifiable information was used and there was no direct involvement of human participants, the requirement for informed consent was waived.

Results

Functional and Clinical Benchmarking of In Silico BAP1 Variant Predictors

Kernel density estimation plots demonstrated partially overlapping but distinct score distributions across all predictors when stratified by SGE classification (Fig 1). Variants designated as pathogenic by SGE (i.e., depleted in functional assays) showed a clear rightward skew, with scores concentrated toward 1.0. In contrast, nonpathogenic variants (classified as unchanged or enriched) were near scores of 0.0. Intermediate score overlap between the 2 classes was observed for all predictors, indicating areas of reduced discriminatory clarity between categories. Combined Annotation Dependent Depletion showed comparatively greater overlap compared to AlphaMissense, REVEL, and MetaRNN. Overall, AlphaMissense showed the clearest separation between classes, demonstrating the best performance.

Figure 1.

Distribution of in silico predictor scores by SGE-derived functional classification of BAP1 missense variants. Kernel density estimation plots display score distributions for AlphaMissense (A), REVEL (B), MetaRNN (C), and CADD (PHRED) (D), stratified by SGE classification. Variants classified as pathogenic by SGE (blue) show right-shifted score distributions relative to nonpathogenic variants (red) across all predictors. AlphaMissense and MetaRNN demonstrate pronounced separation, with pathogenic variants concentrated at higher scores. Rare Exome Variant Ensemble Learner shows moderate separation. Combined Annotation Dependent Depletion exhibits greater distributional overlap between classes. BAP1 = BRCA1-associated protein-1; CADD = Combined Annotation Dependent Depletion; MetaRNN = Meta-predictor based on a Recurrent Neural Network; REVEL = Rare Exome Variant Ensemble Learner; SGE = saturation genome editing.

To quantitatively assess the ability of the predictors to classify BAP1 missense variants in alignment with SGE-derived pathogenicity designations, we performed ROC and PR curve analyses (Fig 2). Fixed literature thresholds were overlaid on ROC and PR curves to illustrate classifier performance at predefined decision points. AlphaMissense achieved the highest area under the receiver operating characteristic curve (0.837; 95% CI, 0.822–0.852), followed by REVEL (0.820; 95% CI, 0.804–0.836), MetaRNN (0.812; 95% CI, 0.795–0.828), and CADD (0.795; 95% CI, 0.778–0.811). Precision–recall analysis demonstrated similar relative performance, with AUPRC values of 0.627 (95% CI, 0.596–0.660) for AlphaMissense, 0.623 (95% CI, 0.591–0.655) for REVEL, 0.621 (95% CI, 0.591–0.653) for MetaRNN, and 0.512 (95% CI, 0.480–0.546) for CADD, all exceeding the baseline pathogenic prevalence of 21.0%.

Figure 2.

Comparative diagnostic performance of in silico predictors for classification of BAP1 missense variants using SGE-derived functional labels. A, Receiver operating characteristic curves for AlphaMissense, REVEL, MetaRNN, and CADD (PHRED). AlphaMissense demonstrated the highest discriminative performance (AUROC = 0.837), followed by REVEL (0.820), MetaRNN (0.812), and CADD (0.795). The dashed diagonal line represents chance-level discrimination. Circular markers indicate classifier performance at predefined literature thresholds. B, Precision–recall curves for AlphaMissense, REVEL, MetaRNN, and CADD (PHRED). AlphaMissense achieved the highest AUPRC (0.627), followed by REVEL (0.623), MetaRNN (0.621), and CADD (0.512). The dashed horizontal line represents the baseline positive-class prevalence (0.21). Circular markers denote performance at the literature recommended pathogenicity thresholds for each predictor. AUPRC = area under the precision–recall curve; AUROC = area under the receiver operating characteristic curve; CADD = Combined Annotation Dependent Depletion; MetaRNN = Meta-predictor based on a Recurrent Neural Network; PR = precision–recall; REVEL = Rare Exome Variant Ensemble Learner; ROC = receiver operating characteristic; SGE = saturation genome editing.

Of the 41 variants reported in the Walpole et al²⁶ germline cohort, 3 were classified as pathogenic. Saturation genome editing data and all predictors were concordant in identifying these 3 likely pathogenic missense variants (p.H94R, p.L100P, and p.Y173C) as pathogenic. Additionally, 4 variants designated as VUS by Walpole et al (p.V29G, p.S98R, p.L180P, and p.W202R) were classified as pathogenic by all predictors and SGE. In contrast, 5 VUS variants (p.N446I, p.H563Q, p.E602D, p.V604M, and p.C649Y) were consistently classified as nonpathogenic by both SGE and all computational predictors. All 34 variants included in our benchmarking analysis are detailed in Table S1 (available at www.ophthalmologyglaucoma.org).

AlphaMissense Prediction Landscape across All Missense Variants

Given its superior discrimination across both ROC and PR analyses, subsequent analyses were focused on AlphaMissense to enable detailed characterization of its performance. Across all 13 851 possible BAP1 missense substitutions, AlphaMissense predictions classified 8694 variants (62.8%) as Likely Pathogenic, 4063 variants (29.3%) as Likely Benign, and 1094 variants (7.9%) as Ambiguous/Uncertain Significance (Fig 3 and Table 1) using the genome-wide defined cutoffs.²⁴ This comprehensive classification served as the foundation for structural mapping and global intolerance profiling.

Figure 3.

Predicted intolerance landscape of BAP1 missense variation. A, AlphaFold2 model (monomer) of the BAP1 protein colored by the mean AlphaMissense pathogenicity score for all 19 possible substitutions at each residue (scale, right). Deep red highlights positions where virtually every amino acid change is predicted to disrupt function; royal-blue segments are predicted to tolerate most substitutions. The deubiquitinase catalytic core and C-terminal interaction motifs cluster in the most intensely red regions (representative pathogenic hotspot indicated by the maroon arrow). In contrast, the long, disordered N-terminal loop is predominantly blue (representative benign region indicated by the blue arrow). B, Distribution of AlphaMissense categorical calls across all 13 851 theoretical BAP1 missense substitutions: Likely Pathogenic 62.8% (red), Likely Benign 29.3% (blue), Ambiguous 7.9% (gray). C, Heatmap of per-residue/per-substitution scores (rows = alternative amino acids, columns = residue number 1–729). Vertical red bands mark residues globally intolerant to change, whereas blue bands mark highly permissive positions. Black cells denote the native amino acid at each position. Together, the structure and matrix reveal that pathogenic missense risk concentrates in the structured catalytic and binding domains, while flexible regions are comparatively tolerant. BAP1 = BRCA1-associated protein-1.

Table 1.

Distribution of BAP1 Missense Variants by Functional, Computational, and Clinical Classifications

SGE Functional Classification	Count of SGE	% of SGE (n = 4619)
Depleted	988	21.4%
Unchanged	3417	74.0%
Enriched	214	4.6%

AlphaMissense Pathogenicity Prediction	Count of SGE	% of SGE (n = 4619)	Count of total (n = 13 851)	% of total (n = 13 581)
Likely pathogenic	2563	55.5%	8694	62.8%
Likely benign	1736	37.6%	4063	29.3%
Ambiguous/uncertain significance	320	6.9%	1094	7.9%

ClinVar Clinical Significance	Count of SGE	% of SGE (n = 4619)	Count of Total (n = 13 851)	% of Total (n = 13 581)
Not in ClinVar	3706	80.2%	12 044	87.0%
Uncertain significance	849	18.4%	1542	11.1%
Conflicting interpretation	52	1.1%	208	1.5%
Pathogenic	6	0.13%	27	0.19%
Benign	6	0.13%	30	0.22%

BAP1 = BRCA1-associated protein-1; ClinVar = Clinical Variant Database; SGE = saturation genome editing.

This table provides a comprehensive overview of BAP1 missense variant annotations across three major dimensions: experimental functional classification using saturation genome editing, computational pathogenicity predictions from AlphaMissense, and clinical significance annotations from ClinVar. For each source, both the subset of experimentally tested variants (n = 4619) and the total set of all possible missense mutations (n = 13 851) are represented where applicable.

Overlaying AlphaMissense pathogenicity onto the AlphaFold structure of BAP1 revealed that residues within the catalytic core and known protein–protein interaction sites consistently showed higher average pathogenicity scores (Fig 3). Conversely, more flexible or disordered regions tended to show lower pathogenicity. The accompanying heatmap highlights residue-level substitution intolerance, and the pie chart summarizes the predicted distribution of likely pathogenic, benign, and ambiguous substitutions (Fig 3).

AlphaMissense Classification of SGE-Tested Subset

Of the 13 851 possible missense substitutions, a total of 4619 (33.3%) were functionally assessed in the SGE data set. Of these, 988 variants (21.4%) were classified as depleted (pathogenic), 3417 variants (73.98%) as unchanged (benign), and 214 variants (4.6%) as enriched (benign) (Table 1).^23,31 AlphaMissense predictions within the SGE-tested subset classified 2563 variants (55.5%) as Likely Pathogenic, 1736 variants (37.6%) as Likely Benign, and 319 variants (6.9%) fell into the Ambiguous/Uncertain Significance category (Table 1). These model-based predictions were used to benchmark against experimentally derived pathogenicity scores.

Gene-Specific Threshold Optimization of AlphaMissense for BAP1

Threshold-specific performance metrics were calculated to explore AlphaMissense diagnostic performance under different classification schemes. Using the Youden index to optimize the balance between sensitivity and specificity, the optimal threshold was determined to be 0.952 (Table 2). At this threshold, AlphaMissense achieved a sensitivity of 0.767 and a specificity of 0.780, with corresponding precision (positive predictive value) of 0.487, negative predictive value of 0.925, F1 score of 0.596, and overall accuracy of 0.777 (Table 3). At this threshold 1556 (33.7%) of the 4619 variants examined by the SGE study were defined as pathogenic. Of the 13 851 possible BAP1 missense variants, AlphaMissense predicted 5816 as pathogenic by the optimized threshold.

Table 2.

Performance Comparison of AlphaMissense Pathogenicity Thresholds and Saturation Genome Editing Annotations for BAP1 Variant Classification

Metric	Youden Threshold (0.952)	AlphaMissense Cutoff (≥0.564/<0.34)
Sensitivity	0.767	0.915
Specificity	0.780	0.496
Precision (PPV)	0.487	0.344
Negative predictive value (NPV)	0.925	0.953
F1 score	0.596	0.500
Accuracy	0.777	0.590

BAP1 = BRCA1-associated protein-1; PPV = positive predictive value.

Youden threshold was empirically derived from the ROC curve to maximize the sum of sensitivity and specificity. AlphaMissense cutoff refers to the predefined pathogenic (≥0.564) and benign (<0.34) thresholds, excluding ambiguous scores.

Table 3.

Confusion Matrices for AlphaMissense BAP1 Variant Classification of Saturation Genome Editing–Annotated Variants Using Different Pathogenicity Thresholds

Threshold	Predicted Nonpathogenic	Predicted Pathogenic
Youden threshold (0.952)
Actual nonpathogenic (TN/FP)	2833	798
Actual pathogenic (FN/TP)	230	758
AlphaMissense cutoff (≥0.564/<0.34, excl. ambiguous)
Actual nonpathogenic (TN/FP)	1655	1681
Actual pathogenic (FN/TP)	82	882

BAP1 = BRCA1-associated protein-1; FN = false-negatives; FP = false-positives; TP = true-positives.; TN = true-negatives.

For the AlphaMissense cutoff, ambiguous scores (0.34 ≤ score < 0.564) were excluded from analysis.

For comparison, we also evaluated AlphaMissense using its predefined classification scheme, which designates variants with scores ≥0.564 as likely pathogenic and <0.34 as likely benign, excluding intermediate “ambiguous” scores from analysis. Under this scheme, sensitivity increased to 0.915, but specificity decreased to 0.496 (Table 2). Precision and F1 score were lower than at the Youden threshold, at 0.344 and 0.500, respectively, while negative predictive value remained high at 0.953. The overall accuracy using the AlphaMissense cutoff was 0.590. A summary of these performance metrics is presented in Table 2.

To further characterize the classification outcomes under each thresholding approach, confusion matrices were constructed and are presented in Table 3. At the Youden threshold of 0.952, the classifier yielded 2833 true-negatives, 798 false-positives, 758 true-positives, and 230 false-negatives. For the AlphaMissense cutoff, analysis was restricted to variants with nonambiguous scores, resulting in 1655 true-negatives, 1681 false-positives, 882 true-positives, and 82 false-negatives. These matrices illustrate the trade-offs between sensitivity and specificity inherent to different thresholding strategies.

ClinVar Classifications and Analysis

Clinical Variant Database annotations for BAP1 missense variants revealed limited clinical characterization across both experimentally assessed and theoretical variants (Table 1). Within the subset evaluated by SGE (n = 4619), the vast majority (3706 variants; 80.23%) were absent from ClinVar. An additional 849 variants (18.38%) carried classifications of uncertain clinical significance, while 52 variants (1.13%) presented conflicting interpretations (Table 1). Only 12 variants (0.26%) in the SGE data set had definitive ClinVar classifications, equally split between pathogenic/likely pathogenic (6 variants) and benign/likely benign (6 variants). Considering the broader context of all possible BAP1 missense substitutions (n = 13 851), a similar pattern was observed: 12 044 variants (86.95%) were not annotated in ClinVar, 1542 (11.13%) were categorized as uncertain, and 208 (1.50%) showed conflicting interpretations (Table 1). Definitive clinical classifications remained rare, covering just 57 variants (0.41%): 27 pathogenic/likely pathogenic and 30 benign/likely benign (Table 1). Among the 57 definitively annotated variants across the whole set, AlphaMissense predictions exhibited a high concordance rate, correctly identifying the clinical significance for 48 variants (84.2%), comprised of 23 pathogenic and 25 benign variants. Within the smaller subset of 12 definitively classified variants from the SGE data set, AlphaMissense achieved an even higher concordance rate, agreeing with ClinVar in 11 cases (91.7%). Only 8 variants (14.0%) across the whole data set were discordant, including 5 variants deemed benign by ClinVar but predicted pathogenic by AlphaMissense, and 3 ClinVar-pathogenic variants classified benign by AlphaMissense.

Discussion

Our study demonstrates that AlphaMissense achieves robust performance in classifying the pathogenicity of BAP1 missense variants, closely aligning with high-quality experimental benchmarks derived from SGE. With an AUC of 0.837 and an AUPRC of 0.628, AlphaMissense substantially outperformed the baseline prevalence of pathogenic variants (21.4%) and showed strong discriminatory ability greater than other tools like CADD, REVEL, and MetaRNN.34, 35, 36, 37 Furthermore, the observed 91.7% concordance with ClinVar annotations highlights AlphaMissense's potential as a valuable computational tool for variant classification. This is further supported by the 100% concordance observed for the 3 likely pathogenic variants reported in the Walpole et al²⁶ germline cohort.

In the broader context of existing literature, AlphaMissense was initially validated genome-wide with approximately 90% precision using a generalized pathogenicity threshold.²⁴ However, as suggested by Cheng et al, the choice of cutoff thresholds can and should be adjusted according to different use cases or specific accuracy trade-offs to achieve desired precision in other labeled data sets. In line with this guidance, our gene-specific analysis for BAP1 revealed a somewhat reduced accuracy of 77.7% using the Youden-optimized threshold (0.952), and 59.0% when applying the AlphaMissense predefined cutoffs (≥0.564 for pathogenic and <0.34 for benign, excluding ambiguous scores) (Tables 2 and 3). This reflects the complex interplay between variant function, structural context, and assay-specific definitions of pathogenicity. Our findings demonstrate the value of recalibrating thresholds for gene-specific contexts to avoid overclassifying pathogenic variants, a limitation previously recognized in computational predictions. Despite this, AlphaMissense reinforced its utility as an effective initial screening tool to rule out benign variants, maintaining excellent negative predictive value across thresholds (92.5% at the Youden threshold; 95.3% at the AlphaMissense cutoff) (Table 2).

Saturation genome editing operationally defines pathogenicity as the depletion of variants that compromise clonal fitness in a haploid cell line viability assay, thereby interrogating strictly cell-autonomous essentiality over a short time horizon.^23,39 Deep mutational-scanning work across diverse proteins shows that such assays excel at detecting catalytic or structural lesions yet frequently overlook tissue-specific or developmental liabilities that manifest only in multicellular organisms.^40,41 Another potential limitation of SGE lies in the scale of high-throughput experimentation, where the reliability of individual variant measurements depends on stringent experimental controls and reproducibility across replicates. By contrast, AlphaMissense integrates long-term evolutionary conservation, three-dimensional structural priors, and population allele frequencies, which are features molded by organism-level selective pressures, and is therefore expected to capture a broader spectrum of clinically relevant dysfunction.^24,42,43 Contemporary guidelines from the American College of Medical Genetics and Genomics and the Association for Molecular Pathology, as well as the Clinical Genome Resource, explicitly recognize this evidence hierarchy, separating well-validated functional assays (classified under criteria PS3 for “Pathogenic Strong 3”and BS3 for “Benign Strong 3”) from computational and evolutionary data streams when assigning pathogenicity codes.^42,43

Viewed through this lens, the seemingly modest concordance between AlphaMissense and SGE for BAP1 (AUC = 0.837; accuracy = 77.7%) should not necessarily be interpreted as model failure. Instead, discordant calls demarcate biological territory where cell-autonomous lethality and organismal fitness decouple. Variants predicted pathogenic by AlphaMissense yet tolerated in the SGE assay often cluster in solvent-exposed regulatory motifs or posttranslational-modification sites that are dispensable for short-term proliferation but critical for chromatin regulation, DNA-damage signaling, or immune surveillance in vivo.^24,39 Conversely, SGE-depleted variants missed by AlphaMissense frequently involve subtle physicochemical changes within the catalytic core that are underrepresented in population databases and therefore receive weaker evolutionary priors. Integrating these orthogonal read-outs, by recalibrating AlphaMissense thresholds with high-confidence SGE data and flagging discordant variants for further study, could transform apparent “errors” into actionable biological insights and materially improve variant classification for genes such as BAP1.

Current clinical prognostic and diagnostic approaches in UM are primarily dependent on DNA-level measurements, including cytogenetic testing for chromosome 3 and 8 changes, and sequencing of BAP1, SF3B1, and EIF1AX.^7,17,44, 45, 46 While these tests are informative, they do not capture the functional consequences of individual variants. AlphaMissense helps bridge this gap by evaluating each BAP1 missense variant both as a genetic alteration and in terms of its predicted effect on protein structure and function. This dual perspective highlights the potential for next-generation prognostic tools that begin with DNA-based detection yet incorporate protein-level interpretation to refine risk stratification. This trend is already evident in the liquid biopsy space, where studies have expanded upon traditional DNA-based approaches. Barwinski et al⁴⁷ demonstrated that tumor-derived DNA in aqueous humor correlates strongly with monosomy 3. Protein-level biomarkers have also been examined in the liquid biopsy setting. Midena et al⁴⁸ showcased that BAP1 and other UM-associated proteins can be detected in aqueous humor using proteomic analysis, suggesting that liquid biopsy may extend beyond DNA to include protein-level biomarkers. Although DNA-based prognostic tools dominate the current landscape, these findings indicate that proteomic analysis also holds practical and clinically meaningful utility. Studies have demonstrated the prognostic value of protein-level analysis in the context of BAP1. In a large cohort, Kennedy et al confirmed that loss of nuclear BAP1 protein expression, as seen by immunohistochemistry, is a powerful prognostic marker, in some cases outperforming American Joint Committee on Cancer staging and cytogenetic testing.⁴⁹

Timing has also been shown to play a critical role in prognostication. Uner et al⁴ demonstrated BAP1 mutations often arise early in tumor progression, at a stage when tumors are only a few millimeters in size. Early sequencing of BAP1 coupled with proteomic interpretation of variants has the potential to improve risk prediction and guide interventions at an earlier clinical stage. Building on this framework, the present study emphasizes the functional impact of missense variants, using AlphaMissense predictions to link DNA-level changes with their protein-level consequences. This integration provides a proof-of-concept for how future prognostic tools may evolve, by combining DNA sequencing with computational proteomic analysis to deliver more accurate and clinically actionable insights.

Clinically, our findings have significant implications for genetic counseling, risk stratification, and personalized surveillance strategies in UM. Given the high proportion of BAP1 variants unobserved (n = 12 044, 86.95%) or currently classified as VUS (n = 1,542, 11.13%) in databases like ClinVar, AlphaMissense provides a rapid and scalable solution to identify variants with high pathogenic potential (Table 1). This may facilitate targeted surveillance strategies and potentially improve early detection of metastasis, optimizing patient outcomes. Furthermore, by identifying variants near decision thresholds (scores between 0.95 and 0.98), our results may guide prioritization for functional validation assays, streamline laboratory workflows, and reduce costs.

Our structural visualization analysis of AlphaFold predicted structures further enhances the interpretability of AlphaMissense predictions, revealing clear patterns of variant intolerance in functionally critical domains, such as the catalytic core and protein-protein interaction sites. This structural viewpoint corroborates previous functional observations that BAP1's critical enzymatic and interaction domains do not tolerate amino acid substitutions without compromising overall protein function.⁵⁰ These insights align with biological expectations, providing biochemical plausibility for computational predictions and further supporting the integration of structural data into variant classification pipelines.

Nonetheless, our study has several limitations. The experimental SGE assay employed a haploid cell line model, which may not fully replicate the complex tumor microenvironment and chromatin context of uveal melanocytes. Additionally, the SGE data set only covers approximately one-third (4619 out of 13 851) of all possible BAP1 missense variants, leaving many rare substitutions untested. Moreover, AlphaMissense thresholds were trained primarily on population-level data, potentially lacking sensitivity to cancer-specific selective pressures. Lastly, the sparse clinical annotations in ClinVar for BAP1 variants limited external benchmarking and potentially contributed to the single discordant variant call observed.

Looking forward, further prospective validation in clinically annotated UM cohorts is essential to confirm the prognostic utility of AlphaMissense predictions. Integrating computational pathogenicity scores with additional genomic and epigenomic data sets, such as gene expression profiles and methylation status, may refine prognostic models and enhance their clinical applicability.⁵¹ Expanding our analytic framework to other critical UM genes, such as GNAQ, GNA11, and SF3B1, could produce a comprehensive variant atlas, promoting standardized variant interpretation across ocular oncology.²²

Conclusion

AlphaMissense demonstrates strong potential for accurately classifying the pathogenicity of BAP1 missense variants in UM, achieving high concordance with both experimental SGE data and ClinVar annotations. By refining gene-specific thresholds and integrating structural insights, our analysis bridges computational predictions with functional evidence, offering a scalable framework for variant interpretation in UM. Importantly, herein we propose an optimized threshold for applying AlphaMissense to BAP1, which improves sensitivity–specificity balance for future variant interpretation studies. These findings support the incorporation of AI-based tools like AlphaMissense into precision oncology workflows, particularly in contexts where clinical annotations are sparse. Future validation in UM patient cohorts and expansion to other key genes will be necessary for translating these insights into clinical utility.

Declaration of Generative AI and AI-Assisted Technologies in the Writing Process

During the preparation of this work, the authors used ChatGPT (OpenAI) and Grammarly AI in order to improve the readability, grammar, and clarity of the manuscript. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

Manuscript no. XOPS-D-25-00884.