Clinical Variant Reclassification in Hereditary Disease Genetic Testing

This cohort study analyzes the rates at which clinical genetic variants are reclassified and the types of evidence that enable reclassification among individuals who undergo genetic testing for hereditary disease.

Key Points

Question

What are the rates, directions, and evidence usage patterns for variant reclassifications in germline genetic testing, and how do these factors indicate accuracy of results and influence population equity?

Findings

Among more than 2 million variants observed in this cohort study of 3 272 035 individuals, 94 453 variants (4.60%) were reclassified over 8 years. Most changed into clinically unambiguous categories, and reclassifications were disproportionately observed in racially, ethnically, and ancestrally underrepresented populations; machine learning–based prediction models contributed to approximately one-half of all reclassifications.

Meaning

These findings suggest that the level of accuracy behind initial variant classifications exceeded certainty thresholds specified by guidelines and that computational approaches dramatically increased reclassifications, affecting large numbers of individuals from racially, ethnically, and ancestrally diverse populations.

Abstract

Importance

Because accurate and consistent classification of DNA sequence variants is fundamental to germline genetic testing, understanding patterns of initial variant classification (VC) and subsequent reclassification from large-scale, empirical data can help improve VC methods, promote equity among race, ethnicity, and ancestry (REA) groups, and provide insights to inform clinical practice.

Objectives

To measure the degree to which initial VCs met certainty thresholds set by professional guidelines and quantify the rates of, the factors associated with, and the impact of reclassification among more than 2 million variants.

Design, Setting, and Participants

This cohort study used clinical multigene panel and exome sequencing data from diagnostic testing for hereditary disorders, carrier screening, or preventive genetic screening from individuals for whom genetic testing was performed between January 1, 2015, and June 30, 2023.

Exposure

DNA variants were classified into 1 of 5 categories: benign, likely benign, variant of uncertain significance (VUS), likely pathogenic, or pathogenic.

Main Outcomes and Measures

The main outcomes were accuracy of classifications, rates and directions of reclassifications, evidence contributing to reclassifications, and their impact across different clinical areas and REA groups. One-way analysis of variance followed by post hoc pairwise Tukey honest significant difference tests were used to analyze differences among means, and pairwise Pearson χ² tests with Bonferroni corrections were used to compare categorical variables among groups.

Results

The cohort comprised 3 272 035 individuals (median [range] age, 44 [0-89] years; 2 240 506 female [68.47%] and 1 030 729 male [31.50%]; 216 752 Black [6.62%]; 336 414 Hispanic [10.28%]; 1 804 273 White [55.14%]). Among 2 051 736 variants observed over 8 years in this cohort, 94 453 (4.60%) were reclassified. Some variants were reclassified more than once, resulting in 105 172 total reclassification events. The majority (64 752 events [61.65%]) were changes from VUS to either likely benign, benign, likely pathogenic, or pathogenic categories. An additional 37.66% of reclassifications (39 608 events) were gains in classification certainty to terminal categories (ie, likely benign to benign and likely pathogenic to pathogenic). Only a small fraction (663 events [0.63%]) moved toward less certainty, or very rarely (61 events [0.06%]) were classification reversals. When normalized by the number of individuals tested, VUS reclassification rates were higher among specific underrepresented REA populations (Ashkenazi Jewish, Asian, Black, Hispanic, Pacific Islander, and Sephardic Jewish). Approximately one-half of VUS reclassifications (37 074 of 64 840 events [57.18%]) resulted from improved use of data from computational modeling.

Conclusions and Relevance

In this cohort study of individuals undergoing genetic testing, the empirically estimated accuracy of pathogenic, likely pathogenic, benign, and likely benign classifications exceeded the certainty thresholds set by current VC guidelines, suggesting the need to reevaluate definitions of these classifications. The relative contribution of various strategies to resolve VUS, including emerging machine learning–based computational methods, RNA analysis, and cascade family testing, provides useful insights that can be applied toward further improving VC methods, reducing the rate of VUS, and generating more definitive results for patients.

Introduction

Human genomes contain substantial germline sequence variation, and distinguishing variants that cause disease from those that are benign is a central requirement in clinical genetic testing for hereditary disease. Accurate and timely variant classification (VC) is critical because variants classified as likely pathogenic (LP) or pathogenic are considered actionable because they can confirm a clinical diagnosis, determine prognosis, guide treatment and prevention strategies, and help estimate risk for family members.^1,2 Each variant observed during genetic testing is classified based on laboratory practice guidelines published by the American College of Medical Genetics (ACMG) and the Association for Molecular Pathology (AMP).³ By assessing various types of evidence, variants are classified into 1 of 5 categories: benign, likely benign (LB), variant of uncertain significance (VUS), LP, or pathogenic. ACMG and AMP guidelines and subsequent updates from ClinGen stipulate certainty thresholds for classification tiers; likely classifications (ie, LP and LB) are considered to have at least 90% certainty of pathogenicity or benignity, terminal classifications (pathogenic and benign) have at least 99% certainty, and VUS classifications occupy the range below 90% certainty.^4,5 Understanding the effectiveness of ACMG and AMP guidelines and the reliability of classification certainty thresholds is a question of enduring interest.^6,7,8

Available evidence during initial variant analysis is often insufficient for an LB, benign, LP, or pathogenic classification. As a result, many newly observed variants are classified as VUS, as we recently described in an analysis of a large clinical cohort.⁹ More than 40% of clinically reported variants are listed as VUS or have conflicting classifications in the ClinVar database, which aggregates VCs from laboratories worldwide.^10,11 Clinical laboratories bear the responsibility to review new evidence as it emerges, reclassify VUS into definitive categories whenever possible, and promptly alert patients and their clinicians.^12,13,14 Beyond this responsibility, some have also actively generated new data and developed new data evaluation methods to reduce VUS.^{15,16,17,18,19}

Despite the importance of VC in clinical genetics, an in-depth analysis of empirical data from a large clinical cohort tested at a clinical laboratory has not been conducted. In this study, we fill this gap by describing the rates and directions of reclassifications and their impact in a diverse population of individuals referred for genetic testing for a range of hereditary disorders. Our study gathers insights into how long initial classifications remain stable, the rates at which variants are reclassified, what proportion of LP or LB variants move to a more definitive state, the types of evidence that enable reclassifications, and reclassification rates across clinical specialties and among race, ethnicity, and ancestry (REA) groups. Understanding these factors can help clinicians appreciate the nuances of VC and reclassifications and provide key insights for laboratories to further improve VC methods.

Methods

Privacy and Ethics

This cohort study was approved by The Western Independent Review Board without the need for individual informed consent because data were deidentified. This study followed the Strengthening the Reporting of Genetic Association Studies (STREGA) reporting guideline for cohort studies.²⁰

Clinical Cohort

The study cohort included individuals referred for genetic testing to Invitae and for whom a clinical report was released between January 1, 2015, and June 30, 2023. Genetic testing was performed using 1 of 517 gene panels precurated by Invitae,⁹ panels customized by ordering clinicians, or exome sequencing. Clinical and demographic data were obtained from clinicians. DNA sequencing, data processing, and clinical reporting were performed as described previously.^21,22 Individuals who requested data deletion or opted out from research were excluded. Data from family members were included only to analyze evidence types contributing to reclassification.

Demographics

Information on patient age, biological sex, and REA groups was provided by ordering clinicians. For REA groups, clinicians selected from preset groups on test order forms: Ashkenazi Jewish, Asian, Black, French Canadian, Hispanic, Native American, Pacific Islander, Sephardic Jewish, White (non-Jewish and non–French Canadian for the purposes of this study), or other. When other was selected, a free-text response could be added. If free-text responses matched a preset group, individuals were included in that group. Individuals with more than 1 reported REA background were grouped as multiracial or multiethnic.

VCs and Reclassifications

DNA variants were classified into 1 of 5 categories (benign, LB, VUS, LP, or pathogenic) using Sherloc, a system based on ACMG and AMP guidelines.^3,7 Variants were excluded if they had a nonstandard classification (eg, pathogenic [low penetrance], increased risk allele, or FMR1 premutation alleles [263 alleles]). To reclassify variants, our laboratory routinely searched for and applied new evidence from published literature, testing family members, updated data in genomics databases, and other sources. An exception to this are LB to benign reclassifications, which are not routinely performed and were therefore simulated (eMethods in Supplement 1). In this study, reclassifications from one end of the VC spectrum to the other (ie, LB or benign to LP or pathogenic or vice versa) were referred to as reversals. To determine whether the quantity and direction of evidence for a given VUS was associated with the rate and direction of eventual reclassification, we calculated summarized evidence scores (eMethods in Supplement 1). All variants analyzed in this study were deposited into the National Institutes of Health ClinVar database.

Statistical Analysis

One-way analysis of variance followed by post hoc pairwise Tukey honest significant difference tests were used to analyze differences among means. Pairwise Pearson χ² tests with Bonferroni corrections were used to compare categorical variables among groups. Because REA categories are known to affect VC,^9,23 reclassification rates in each REA category (other than White) were compared with the White category, which represented the largest proportion of the clinical cohort. Statistical analyses were performed using R software version 4.3.0 (R Project for Statistical Computing). All significance tests were 2-tailed, and a corrected (if applicable) P < .05 was considered statistically significant. Data were analyzed from July 2023 to February 2024.

Results

The cohort comprised 3 272 035 individuals (median [range] 44 [0-89] years; 2 240 506 female [68.47%] and 1 030 729 male [31.50%]). The largest proportions of individuals were reported as White (1 804 273 individuals [55.14%]), Hispanic (336 414 individuals [10.28%]), or Black (216 752 individuals [6.62%]) (eTable 1 in Supplement 1). From this cohort, we reported 2 051 736 variants observed over an 8.5-year period. Nearly one-half (1 100 464 variants [53.64%]) were initially classified as VUS and 821 089 (40.02%) as either LB or benign. The initial classification changed to another category (ie, reclassification) for 94 453 variants (4.60%). Some variants were reclassified more than once during the study period, resulting in 105 172 total reclassification events. While changes between every combination of classification categories were observed, the propensity to undergo reclassification varied among these categories. Likely pathogenic variants were reclassified most frequently, followed by VUS, while benign, LB, and pathogenic variants were reclassified in much smaller proportions (Table 1).

Table 1. Variant Reclassification Rates^a.

Initial classification	Variant classifications, No. (%) (N = 2 062 273)	Unchanged variants at the end of the study period, No./total No. (%)	Reclassified variants at the end of the study period, No./total No. (%)	Reclassifications, No. (%) (N = 105 172)
Benign	142 483 (6.91)	142 317/142 483 (99.88)	166/142 483 (0.12)	166 (0.16)
Likely benign	687 226 (33.32)	653 835/687 226 (95.14)	33 391/687 226 (4.86)	33 391 (31.75)
Variant of uncertain significance	1 100 487 (53.36)	1 035 647/1 100 487 (94.11)	64 840/1 100 487 (5.89)	64 840 (61.65)
Likely pathogenic	30 214 (1.47)	23 652/30 214 (78.25)	6562/30 214 (21.72)	6562 (6.24)
Pathogenic	101 863 (4.94)	101 650/101 863 (99.79)	213/101 863 (0.21)	213 (0.20)

^a The numbers in this table represent classification events. Because very few variants underwent reclassification more than once (approximately 10 000 of more than 2 million), the number of reclassification events approximates the number of reclassified variants.

Empirical Estimates of VC Accuracy

The overall classification stability, as well as reclassification direction and rate, were evaluated to derive empirical estimates of the accuracy of the initial classification. Taking stability over time as a measure, we observed that virtually all benign classifications (142 317 of 142 483 events [99.88%]) and pathogenic classifications (101 650 of 101 863 events [99.79%]) remained unchanged, exceeding the 99% certainty threshold recommended for these 2 classification categories by the ACMG and AMP guidelines and ClinGen (Table 1). Furthermore, the percentage of all LP variants that remained LP or were reclassified to pathogenic at the end of the study period was 99.30% (30 003 of 30 214 LP classifications), and the comparable percentage for LB variants was 99.98% (687 092 of 687 226 LB classifications). However, stability is an imperfect measure of accuracy because some variants may not have had sufficient time for new evidence to confirm or reverse the initial classification.

We also examined the direction of reclassification for variants initially classified in the likely (ie, LB and LP) categories (eFigure in Supplement 1). Specifically, when new evidence triggered their eventual reclassification, how often were initial classifications confirmed (ie, LB reclassified to benign or LP reclassified to pathogenic) vs reversed (ie, from benign or LB to pathogenic or LP and from pathogenic or LP to benign or LB)? To answer this, we calculated the proportions of classification confirmations vs reversals in our empirical data and compared them with the certainty thresholds (>90%) prescribed by ACMG and AMP guidelines. Among LP and LB variants that were either confirmed or reversed, 99.89% of LP classifications (6351 of 6358 classifications) were confirmed as pathogenic, and 99.89% of LB classifications (33 257 of 33 293 classifications ) were confirmed as benign. Taken together, while the ACMG and AMP guidelines set a mean probability of pathogenicity certainty target of 94.5% for LP classifications (ie, the midpoint of the bounding 90% and 99% thresholds) and 5.5% for LB classifications, our data instead indicated more stringent corresponding mean (SD) values of 99.88% (3.46%) and 0.11% (3.31%), respectively.

Of 6562 LP reclassifications, 6351 (96.78%) were reclassified to pathogenic among all possible directions (ie, to pathogenic, VUS, LB, or benign), whereas the comparable rate of reclassification of LB to benign was 99.60% (33 257 of 33 391 reclassifications) (Table 2). These reclassification rates define the lower boundaries of the empirical estimates of LP and LB accuracy because reclassification to VUS neither confirms nor rejects the initial assertion. Finally, only 663 of 105 172 reclassifications (0.63%) shifted toward less certainty (ie, from benign, LB, LP, or pathogenic to VUS; pathogenic to LP; or benign to LB) and 61 of 105 172 (0.06%) reversed from one end of the VC spectrum to the other (ie, pathogenic or LP to benign or LB or vice versa).

Table 2. Variant Reclassification Direction and Rates^a.

Original classification	After reclassification, No. of classification events/total No. of reclassifications (%)
	Benign	Likely benign	VUS	Likely pathogenic	Pathogenic
Benign	NA	141/166 (84.94)	12/166 (7.23)	3/166 (1.81)	10/166 (6.02)
Likely benign	33 257/33 391 (99.60)	NA	98/33 391 (0.29)	13/33 391 (0.03)	23/33 391 (0.07)
VUS	10 356/64 840 (15.97)	40 730/64 840 (62.82)	NA	7996/64 840 (12.33)	5758/64 840 (8.88)
Likely pathogenic	2/6562 (0.03)	5/6562 (0.08)	204/6562 (3.11)	NA	6351/6562 (96.78)
Pathogenic	2/213 (0.94)	3/213 (1.41)	188/213 (88.26)	20/213 (9.39)	NA

Abbreviations: NA, not applicable; VUS, variant of uncertain significance.

^a The numbers in this table represent classification events. Because very few variants underwent reclassification more than once (approximately 10 000 of more than 2 million), the number of reclassified variants approximates the number of total reclassification events.

Among the clinical areas in which genetic testing was performed, the proportion of variants that underwent reclassification ranged from 18 789 of 564 903 variants (3.33%) to 6947 of 77 535 variants (8.96%) (eTable 2 in Supplement 1). The patterns of classification stability, likely classification (ie, LB or LP) confirmation rates, and the rate of reclassifications to VUS or reversals were consistent among each of 12 clinical specialties in which large numbers of individuals underwent genetic testing for diverse hereditary disorders (eTables 3-5 in Supplement 1).

VUS Reclassifications and the Causes of Reclassification

Reclassification of VUS accounted for 64 840 of the 105 172 reclassification events (61.65%) (Table 1 and the eFigure in Supplement 1), with the majority being VUS to benign or LB changes (51 086 of 64 840 reclassifications [78.79%]) (Figure 1A). These reclassifications were due to 3 different types of evidence (Figure 1B). First, approximately 30% of VUS reclassifications (19 414 of 64 840 reclassifications [29.94%]) were due to new data obtained organically, including genetic data added to public databases (eg, gnomAD) and clinical observations drawn from both publications and new patients tested at our laboratory. Reclassifications due to new clinical observations accounted for the largest proportion of VUS reclassifications to LP or pathogenic (4617 of 13 754 reclassifications [33.57%]). Second, 11.14% of VUS reclassifications (7221 of 64 840 reclassifications) were due to new data actively generated in our laboratory explicitly for the purpose of reclassifying VUS, either through variant segregation analysis in family members or RNA testing. One-quarter of family studies (5899 of 23 418 studies [25.19%]) led to reclassification of at least 1 VUS. Nearly three-quarters of reclassifications (5011 of 6713 reclassifications [74.65%]) changed to LB or benign, and the remainder to LP or pathogenic. However, only 11.83% of eligible families (23 418 of 197 900 families) chose to pursue testing. Separately, results from RNA splicing analysis contributed to 508 of 64 840 (0.78%) VUS reclassifications. Finally, 57.18% of reclassifications (37 074 of 64 840 reclassifications) occurred because of improvements in evidence assessment methods, namely through implementation of novel machine learning-based, gene-specific in silico models validated in our laboratory for (1) establishing precise population allele frequency thresholds to classify benign variants, (2) predicting functional effects including algorithms targeting missense variants and algorithms measuring the informativeness of high-throughput multiplex assays of variant effects,²⁴ or (3) predicting the effects on RNA splicing.²⁵

Figure 1. Direction and Sources of Evidence for Variant of Uncertain Significance (VUS) Reclassifications.

A, Final classification for variants that were initially classified as VUS. B, Three broad categories provided additional evidence to support reclassifications: improved use of available data (black), data obtained organically from internal and external sources (dark gray), and data actively generated by additional testing (light gray). C, Final classification of variants initially classified as VUS, stratified by the level of evidence toward pathogenic and benign available at the time of initial classification. Each horizontal bar represents different categories of summarized evidence scores (eg, a collection of variants where the combined score = 1); individual evidence supporting pathogenic or benign classifications are captured using a numeric point-based system with scores between −5 and 5 at 0.5 point intervals, with a higher score indicating more evidence supporting pathogenic and a lower score indicating more evidence supporting benign. Scores between −3 and 4 (not inclusive) are classified as VUS. B indicates benign; LB, likely benign; LP, likely pathogenic; ML, machine learning; P, pathogenic.

Because the variants were classified using Sherloc, a numeric point-based implementation of the ACMG guidelines, we examined the rates and direction of VUS reclassifications when the VUS were subcategorized based on the amount and direction of evidence criteria already applied (Figure 1C). Overall, we observed a clear trend where VUS with more pathogenic evidence were more likely to be reclassified as LP or pathogenic, while VUS with more benign evidence were more likely to be reclassified as LB or benign. Notably, the trend between the summarized evidence score and the rate of reclassifications to LP or pathogenic and LB or benign did not track perfectly.

Changes in VUS Reclassification Rates

The number of VUS reclassifications steadily increased over time, with most (51 260 of 64 752 reclassifications [79.16%]) occurring in the final 2 years of the study period (2021 to 2023) (Figure 2A). While this increase, in part, reflects the expansion of known variants year over year, normalization of reclassification counts by cumulative VUS observed indicated a steady increase in VUS reclassification rate between 2015 and 2021 and a rapid acceleration starting in 2021 (Figure 2B). This acceleration coincided with the incorporation of data from machine learning–based models into VC and also with updates to gnomAD. Despite this improvement in VUS reclassifications, most VUS first observed in 2022 (249 489 of 253 932 classifications [98.25%]) were unchanged a year later and approximately two-thirds of VUS first observed in 2015 (3074 of 4690 classifications [65.54%]) were still not resolved at the end of the study (Figure 2C).

Figure 2. Variant of Uncertain Significance (VUS) Reclassification Rates and Classification Reversal Rates.

A, Cumulative VUS reclassifications over time. B, Cumulative VUS reclassifications over time normalized by the total number of VUS. C, Time to VUS reclassification. D, The combined rate of classification reversals and reclassifications to VUS among all reclassifications decreased 4-fold by the end of the study period. B indicates benign; LB, likely benign; LP, likely pathogenic; P, pathogenic.

Reversals in Classification and Reclassifications to VUS

Reversals in classification (ie, pathogenic or LP to benign or LB or vice versa) were exceedingly rare (Table 2 and the eFigure in Supplement 1). Of the 61 reversals observed among 961 786 pathogenic, LP, benign, or LB variants (rate of 0.0006), 12 changed from pathogenic or LP to benign or LB and 49 from benign or LB to pathogenic or LP. In addition to these reversals, 393 changed from pathogenic or LP to VUS and 84 from benign or LB to VUS (eFigure in Supplement 1). These reclassifications were attributable to 4 causes: new data, gene curation, errors, and review of original data (eTable 6 in Supplement 1). In contrast with the acceleration of VUS reclassifications noted previously, the normalized rate of reversals and reclassifications to VUS decreased by 4-fold, from a rate of 0.0150 in 2015 to 0.0038 in 2023 (Figure 2D). This ratio generally declined year over year, except in 2019 when the discovery of a biologically relevant cryptic exon in the VHL gene led to reclassification from LB to VUS for 13 variants previously assumed to be intronic.²⁶

Patterns of VUS Reclassifications in REA Groups

The cumulative number of individuals who received 1 or more VUS reclassifications markedly increased after 2018 (Figure 3A), likely due to yearly increases in the number of individuals tested between 2018 to 2023 and the improved evidence-generating methods described earlier. Although this increase benefited individuals from all REA groups, 172 752 White individuals received reclassifications by the end of the study period, the highest among all REA groups, likely because they represented the majority of individuals referred for genetic testing. However, when normalized by the number of individuals in each REA group, Sephardic Jewish individuals represented the highest proportion (2906 of 7858 individuals [36.98%]) of those who received a VUS reclassification, while French Canadian individuals (689 of 3481 individuals [19.79%]), Native American individuals (820 of 4092 individuals [20.04%]), and White individuals (172 752 of 807 873 individuals [21.38%]) represented the lowest. Mean (95% CI) reclassified VUS per individual also differed significantly among REA groups, from 0.51 (0.50-0.53) VUS reclassified in Sephardic Jewish individuals to 0.22 (0.20-0.23) in French Canadian individuals, 0.25 (0.23-0.27) in Native American individuals, and 0.26 (0.26-0.27) in White individuals (Figure 3B).

Figure 3. Patterns of Variant of Uncertain Significance (VUS) Reclassifications in Race, Ethnicity, and Ancestry (REA) Groups.

A, Trends in cumulative VUS reclassifications among REA groups over the study period. B-C, Normalized VUS reclassification rates among REA groups at the end of the study period. In panel C, the bars represent the 95% CI of the mean reclassified VUS per patient in each ethnicity group.

^aP < .001 vs the comparison group (White).

Discussion

In this cohort study, the empirically estimated accuracy of pathogenic, LP, benign, and LB classifications exceeded the certainty thresholds set by current VC guidelines, suggesting the need to reevaluate definitions of these classifications. Improvements to VC methods are an area of intense activity in hereditary disease genetic testing. This is driven both by the accumulation of data from millions of individuals around the world and by the clinical costs of uncertain results, which are not actionable and therefore impede the ability of clinicians to use genetic information to manage the health of their patients. Given the convergence of large genomics datasets (in ClinVar, gnomAD, and others), data sharing among clinical laboratories and research groups, and the development of sophisticated computational models, it has been optimistically speculated that the medical genetics community will eliminate VUS within the next decade.¹⁰

Substantial numbers of VUS can be resolved with additional clinical information or through variant segregation analysis in family members.⁹ However, the rate at which this occurs is still low,²⁷ calling for greater collaboration between clinical laboratories and referring clinicians. For instance, family variant testing in our laboratory—offered at no additional cost—is pursued in just over 10% of eligible cases, thus depriving many patients of potentially actionable results. This shortfall may be partly improved by empowering patients to understand the importance of collecting more detailed information by organizing family members for DNA studies and by empowering clinicians who are not experts in clinical genetics to identify individuals that may benefit from testing (eg, via partnerships with genetic counselors or using tools such as chatbots).^28,29

Because of the paucity of data that can serve as evidence during VC, the number of observed VUS has been increasing as more individuals undergo genetic testing, and we may expect this trend to continue given that populations historically underrepresented in genetic testing likely harbor a substantial proportion of rare VUS. Emerging computational and in vitro methods designed to predict the effects of DNA variants are proving valuable in addressing this challenge by generating new genetic insights at scale and supporting reclassification of VUS into LP or pathogenic or, more often, into LB or benign classifications.^30,31,32,33 This study illustrates the magnitude of improvement in VC outcomes made possible by applying these types of methods. Moreover, these methods have a greater impact on resolving VUS in underrepresented populations because of their population-agnostic approach to assessing the impact of DNA variants, as highlighted in this study. These findings underscore the growing importance for VC guidelines to include better guidance on how to standardize the development and validation of computational methods for pathogenicity predictions. Recent work by Pejaver et al³⁴ made important advances on this issue by providing guidance on validation and usage of these types of algorithms. However further guidance is warranted on various additional topics including (1) situation-specific (eg, gene-specific, disease-specific, and molecular mechanism-specific) algorithm evaluations, (2) best-practice guidelines for developers on how to build algorithms that are most compatible with VC systems, and (3) how to rationally combine different types of evidence within a quantitative framework while maintaining interpretability (ie, not black box).

When the ACMG and AMP guidelines were drafted almost a decade ago,³ and when subsequent updates from ClinGen working groups were made,^4,5 there were inadequate tools and data to calibrate the accuracy of VCs. It was therefore not possible to determine whether the certainty of classifications made based on ACMG and AMP guidelines faithfully matched the defined thresholds. Results from this study encourage the clinical genomics community to revisit the 90% certainty thresholds for LP and LB classifications. Given that clinicians often treat LP classifications as actionable, the current certainty threshold of 90% essentially allows for an arguably high rate of false positive genetic testing results (ie, a 1 in 10 chance of an LP variant not being disease-causing). The retrospective observation that 99.9% of LP variants that were reclassified into definitive categories were actually resolved as pathogenic variants suggests that the stated level of certainty required for an LP classification should be increased to better align with current clinical practice.

Limitations

This retrospective study has limitations that are typical of empirical data from clinical genetic testing laboratories, including possible inherent biases in the number of genetic tests performed within clinical areas and the diversity of individuals referred for genetic testing. Furthermore, the patterns and conclusions reported here are drawn from a single laboratory and are thus a product of our specific VC processes and include some proprietary data. Moreover, because VC itself is an ever-shifting landscape, the patterns reported here may not carry forward. While we do not expect these factors to change any of the key observations of the study, similar analyses of variant reclassification results from other clinical genetic testing laboratories could provide a fuller understanding of the current state of variant reclassifications.

Conclusions

In this cohort study of 3 272 035 individuals, we tracked changes in VCs over time for more than 2 million genetic variants. We observed high stability for variants classified as pathogenic, LP, LB, and benign, suggesting an overall very high accuracy in these classifications. Importantly, the observed accuracy exceeded the target thresholds set in the ACMG and AMP guidelines; this gap between observed and expected accuracy has implications for which patients do or do not have clinical management altered based on genetic test results. However, we also observed discouragingly high stability in variants initially classified as VUS, with fewer than 6% of such variants being resolved within the study period. However, the rate of VUS reclassifications has accelerated substantially in recent years, a change driven primarily by recent advances in machine learning–based algorithms that leverage large accumulations of variant data in the public domain and then quantitatively assess the value of each dataset. These findings offer hope that further acceleration of VUS reduction is possible through advances in these rapidly evolving technologies. Furthermore, these analysis platforms likely represent initial steps toward a clinically valid, quantitative VC system that has been a long sought-after goal in clinical molecular genetics. To continue improving VC, the rate of reclassifications, and to effectively convey to clinicians the degree of certainty associated with VC, future improvements to VC guidelines must include better guidance on how to standardize the development and validation of computational methods for variant effect predictions, quantitatively assess evidence, and rationally combine different types of evidence within a quantitative framework.

Supplement 1.

eMethods.

eTable 1. Characteristics of Clinical Cohort

eFigure. Proportions of Reclassification Events by Type

eTable 2. Proportions of Variants Reclassified by Clinical Area

eTable 3. Variant Reclassification Rates Across Clinical Areas

eTable 4. Measurement of Accuracy of Likely Pathogenic and Likely Benign Classifications Across Clinical Areas

eTable 5. Proportion of Variants That Were Reclassified Into Different Categories Across Clinical Areas

eTable 6. Reasons for Very Rare Variant Classification Reversals

Supplement 2.

Data Sharing Statement