A survey assessing adoption of the ACMG-AMP guidelines for interpreting sequence variants and identification of areas for continued improvement

In March of 2015, a workgroup of the American College of Medical Genetics and Genomics (ACMG) and the Association of Molecular Pathology (AMP) published standards for the Interpretation of Sequence Variants.¹ These guidelines recommended a systematic scoring process for analyzing evidence and classifying variants into five tiers. In addition, they recommended the following terminology for sequence variant interpretation in Mendelian genes: pathogenic (P), likely pathogenic (LP), uncertain significance (VUS), likely benign (LB), and benign (B). If pathogenicity is asserted, these guidelines advised laboratories to specify condition and mode of inheritance.

While these guidelines have been widely referenced by clinical laboratories, the adoption of these guidelines and how laboratories are incorporating them into variant interpretation workflows has not been assessed. To evaluate the utilization of the ACMG-AMP terminology and scoring process, the NIH-funded Clinical Genome Resource surveyed clinical testing laboratories registered in the Genetic Testing Registry (GTR: https://www.ncbi.nlm.nih.gov/gtr/).² In July 2017, an invitation email was sent to one representative from each of the 195 US-based and 170 international laboratories. The survey assessed whether or not a laboratory had adopted the terminology and scoring system recommended in the ACMG-AMP guidelines, as well as reasons why or why not. Representatives from 65 laboratories responded, including individuals from 33 international laboratories representing 15 different countries. Respondents from Canada (n=9), UK (n=4), and Australia (n=4) accounted for the highest representation among international laboratories.

Analysis of survey responses demonstrated high adoption of the ACMG-AMP terminology recommendations. 95% of surveyed laboratories (62/65 respondents) reported using the ACMG-AMP five tiers for classifying variants in Mendelian genes. International laboratories were just as likely to report using the guidelines as US-based laboratories. 86% of surveyed laboratories classified variants with respect to a condition and mode of inheritance in a uniform and structured manner, and 89% of laboratories routinely avoided the use of ‘mutation’ or ‘polymorphism’ in clinical reports in favor of ‘variant.’

The ACMG-AMP guidelines recommended that laboratories use at least the five classification tiers (P, LP, VUS, LB, B), but allowed laboratories to optionally use more tiers. While 78% (51/65) of responding laboratories use just the five classification tiers, 22% of laboratories (14/65) indicated that they use additional terms, such as the use of subcategories (eg, VUS - favor benign), to further classify variants. Examples of these subcategories provided by respondents included: “uncertain clinical significance, possibly pathogenic” and “topline VUS.” It should be noted that this survey did not delineate between the use of subcategories within a laboratory’s internal databases versus what is reported or submitted to ClinVar, which currently does not accept these subcategories (ClinVar: https://www.ncbi.nlm.nih.gov/clinvar/).

In regards to the evidence criteria provided in the ACMG-AMP guidelines (Figure 1),¹ 36% of laboratories (n=22) use the evidence criteria exactly as described, 44% (n=28) use an approach that is roughly consistent, and 17% (n=11) use an approach that they considered more advanced than the ACMG-AMP approach - for a total of 97% of laboratories (62/64) using approaches that they considered consistent with these professional guidelines. The fact that many laboratories have felt the need to modify or evolve the criteria is consistent with reports that the criteria may be inconsistently applied across groups.³ As an example of groups evolving the guideline, Nykamp and colleagues reported implementing a refined version of the guideline to create more consistency and accuracy in the application of the guideline.⁴ Jarvik and Browning proposed additional guidance for quantifying segregation, and Walsh and colleagues provided guidance on the use of evidence from tumors.^5,6 In addition, ClinGen’s Sequence Variant Interpretation Working Group has published several papers and posted additional guidance around many criteria, with more under development (https://www.clinicalgenome.org/working-groups/sequence-variant-interpretation/).^7,8,9 Furthermore, ClinGen’s Variant Curation Expert Panels are publishing specifications of the guidelines to further clarify when certain criteria are applicable for a given gene or disease and what thresholds and strengths of evidence are appropriate for certain criteria.^{10,11,12,13,14,15}

This survey also identified that the community would benefit from consensus on how to annotate low penetrance variants in genes for Mendelian disorders. 48% (30/63) of laboratories responded that they describe such variants using the standard five-tier terminology, but add qualifiers to denote low penetrance or a mild effect (eg, pathogenic, low penetrance). 14% of responding laboratories described such variants as ‘risk alleles’ with a confidence level, 13% of laboratories used an alternate terminology system, and 25% of laboratories did not have a specific terminology system in place for such variants. These results are consistent with a publication by Yang and colleagues that reports that a large source of discordance in ClinVar is due to variants with low penetrance.¹⁶ In response to this area of needed guidance, ClinGen has launched a task team that is beginning to develop further recommendations for the classification of low penetrance variants.

Finally, continued efforts are needed to encourage laboratories to submit to ClinVar given the important role of data sharing in improving consistency and accuracy of variant interpretation. As of July 2017, only 11% (7/63) of respondents regularly submitted at least 75% of their variants to ClinVar once a year. 33% (21/63) submitted in the past but did not do so regularly or have only submitted a limited set of variants, 29% of laboratories had never submitted but hoped to start within the next year, and 27% did not submit at all. It has become clear that no matter how detailed guidelines are for variant interpretation, laboratories do not always interpret variants consistently, either due to varied application of evidence criteria or the use of alternate sources or unpublished evidence.^3,17 Yet sharing within ClinVar has enabled resolution of the majority of differences assessed, emphasizing the critical importance of laboratory submission to ClinVar as a component of consistency in the application of variant interpretation guidelines.^18,9,17 To promote recognition of laboratories that engage in data sharing and interpretation discrepancy resolution efforts, ClinGen has published a list of laboratories (www.clinicalgenome.org/lablist) that meet minimum requirements for data sharing. This list can be referenced by care providers and payers to guide their decision making on where to order genetic testing and when to reimburse testing. Interested clinical testing laboratories are encouraged to contact ClinGen to participate in these important data sharing activities.

In summary, the ACMG-AMP guidelines provided an evidence-based approach for classifying and interpreting sequence variants. The survey described here demonstrated widespread adoption of the recommendations in just three years. However, the guidelines noted that further refinement and disease specifications would be needed to continue to improve the accuracy and consistency of variant interpretation and our survey has confirmed this need for ongoing development. Future efforts should focus on the refinement of criteria, development of consensus guidance for how to annotate low penetrance variants in genes for Mendelian disorders, recommendations of standard terms for subcategories of variants of uncertain significance, and methods to incentivize widespread deposition of variant interpretations into ClinVar.