Summary
Sharing genomic variant interpretations across laboratories promotes consistency in variant assertions. A landscape analysis of Australian clinical genetic-testing laboratories in 2017 identified that, despite the national-accreditation-body recommendations encouraging laboratories to submit genotypic data to clinical databases, fewer than 300 variants had been shared to the ClinVar public database. Consultations with Australian laboratories identified resource constraints limiting routine application of manual processes, consent issues, and differences in interpretation systems as barriers to sharing. This information was used to define key needs and solutions required to enable national sharing of variant interpretations. The Shariant platform, using both the GRCh37 and GRCh38 genome builds, was developed to enable ongoing sharing of variant interpretations and associated evidence between Australian clinical genetic-testing laboratories. Where possible, two-way automated sharing was implemented so that disruption to laboratory workflows would be minimized. Terms of use were developed through consultation and currently restrict access to Australian clinical genetic-testing laboratories. Shariant was designed to store and compare structured evidence, to promote and record resolution of inter-laboratory classification discrepancies, and to streamline the submission of variant assertions to ClinVar. As of December 2021, more than 14,000 largely prospectively curated variant records from 11 participating laboratories have been shared. Discrepant classifications have been identified for 11% (28/260) of variants submitted by more than one laboratory. We have demonstrated that co-design with clinical laboratories is vital to developing and implementing a national variant-interpretation sharing effort. This approach has improved inter-laboratory concordance and enabled opportunities to standardize interpretation practices.
Sharing genomic variant interpretations across laboratories promotes consistency. The Shariant platform was developed to enable ongoing sharing of variant interpretations and associated evidence, resolution of inter-laboratory discrepancies, and streamlined submission of variant assertions to ClinVar. This approach has improved concordance and enabled opportunities for standardization of practices between Australian laboratories.
The benefits of sharing genomic variant interpretations and associated evidence across laboratories is widely recognized; it allows for improved diagnostic accuracy and patient management.1,2 One key benefit is the promotion of consistency through the identification and resolution of variant-classification discrepancies: sharing variant interpretations resolved 62% of discrepancies across 41 laboratories submitting to ClinVar3 and 31% of discrepancies across 12 laboratories participating in the Canadian Open Genetics Repository.4 As a result, several professional bodies, including the American College of Medical Genetics and Genomics, have incorporated the concept of variant sharing into best-practice guidelines.5
Within Australia, clinical genetic-testing laboratories undertaking germline variant curation encompass both the public (usually state-based and centered around major hospitals and genetics clinics) and private sector. It is mandatory for Australian laboratories to have a policy for submitting variants to relevant clinical databases,6 and laboratories are encouraged to submit genotypic data to clinical databases as part of their accreditation.6,7 Despite this recommendation, in July 2017 fewer than 300 variants had been submitted to ClinVar8 by Australian laboratories, a finding corroborated by a national survey reporting that 75% of laboratories did not submit to international databases.9 These observations indicated that curation knowledge was siloed in individual laboratories, and the Royal College of Pathologists of Australasia identified lack of genetic-data sharing as an area of concern.9 As a result of the state and territory-based nature of Australian healthcare, compounded by the operation of private laboratories, between-laboratory classifications for the same variant can differ. This can lead to inequities in patient counseling and management, which are particularly problematic where results differ for individuals from the same family.
Australian Genomics is a national research initiative aimed at the implementation of genomic medicine into routine clinical practice.10 As part of a specific project tasked to standardize variant interpretation, representatives of Australian clinical genetic-testing laboratories were consulted iteratively so that barriers to variant sharing in general could be identified and so that key requirements for sharing variant interpretations nationally could be defined. This consultation drove the development of Shariant: a controlled-access platform designed to simplify sharing of variant interpretations and associated evidence for Australian laboratories while minimizing disruption to their current workflows. Here we describe the evolution and implementation of Shariant and present initial data demonstrating its benefit for diagnostic accuracy.
Consultation
Landscape analysis
Australian clinical testing laboratories were surveyed from November 2016 to February 2017 so that current genetic-testing practices could be mapped (see supplemental material and methods). Responses (full or partial) were received and collated for 22/30 laboratories conducting genetic testing at the time of the survey. Twenty reported conducting germline testing in some capacity, and only five performed exome and/or genome sequencing. Most responding laboratories (81%) used the American College of Medical Genetics and Genomics and Association for Molecular Pathology (ACMG/AMP) guidelines,11 either as published or with modifications, for classification of germline variants. Somatic testing was conducted by 11 responding laboratories at the time of the survey, and there was no clear consensus on interpretation guidelines for somatic variants.
This evaluation of variant-interpretation practices across Australian clinical genetic-testing laboratories identified areas that had already achieved consensus (e.g., use of ACMG/AMP guidelines for germline variant classification) or would benefit from consultation and standardization (e.g., lack of standards to interpret somatic variants), and it thus informed design of a national variant-sharing platform.
Workshop: Variant-interpretation sharing
A workshop was then conducted at the Human Genetics Society of Australasia conference in August 2017. Representatives from 15 Australian laboratories discussed their willingness to share variant interpretations with the ClinVar database, identified software being used to store variant interpretations, and discussed the potential for variant interpretation sharing.
Consultation with laboratories identified that most respondents were willing to share variant interpretations but wished to seek advice on ethico-legal considerations from their organization before providing commitment to share. The key barriers to sharing between laboratories, and additional incentives that would increase the willingness to share, are shown in Tables 1 and 2, respectively. Solutions to these barriers, and inclusion of incentives, formed the basis for requirements for a national variant-interpretation sharing platform.
Table 1.
Barriers to sharing and solutions implemented by Shariant
| Barrier to sharing | Solution |
|---|---|
| Resources | |
| There is no time to manually prepare data for upload to existing databases. | automated connection to laboratory interpretation system |
| There is limited bioinformatic expertise. | Shariant developer to support integration with laboratory interpretation systems; automated mapping and data transformation of exportable evidence |
| Consent | |
| What information can be shared and with whom? | controlled-access platform; laboratories decide on extent of (clinical) data to be shared |
| Other | |
| Interpretation tools differ between laboratories and can change over time. | sharing agnostic to interpretation system/s; flexibility in connection solutions; work with interpretation system vendors to improve connection |
| “Just another (static) database to check.” | “real-time” connection from laboratory interpretation system allows viewing variants submitted by other laboratories nationally |
Table 2.
Incentives to share and solutions implemented by Shariant
| Incentives to share | Solution |
|---|---|
| storage of sufficient evidence to allow review and re-use of existing curations | submission of structured evidence against ACMG/AMP guidelines |
| identification and resolution of classification discrepancies prior to international sharing | discrepancy-resolution tooling |
| streamlined submission to ClinVar | automated formatting to ClinVar specifications and submission via the ClinVar submission API |
Evaluation of tools and platform selection
Nine existing tools (names not disclosed), including commercial and non-commercial as well as national and international, underwent preliminary evaluation so that the potential to share variant interpretations and associated evidence could be assessed (see supplemental material and methods). The preferred tool was selected on the basis of pre-existing functionality against the evaluation framework (Table S1), costs required for software development, technical advantages and limitations, and licensing models. VariantGrid, a tool developed at the Centre for Cancer Biology (an SA Pathology and University of South Australia Alliance), was chosen as the basis for development of the national variant-interpretation sharing solution now known as Shariant.
Shariant implementation and features
Shariant operations were approved by the Melbourne Health Human Research Ethics Committee (HREC/16/MH/251) and QIMR Berghofer Medical Research Institute Human Research Ethics Committee (P3447).
Formal evaluation of VariantGrid identified key functionalities that required development or improvement to meet the requirements of Shariant as informed by laboratory consultations: controlled access; extension of the application programming interface (API) to accept structured evidence; ability to accept and liftover variants between GRCh37 and GRCh38; and identification and resolution of classification discrepancies. These functionalities were developed over nine months. In parallel with this initial development, a detailed “terms of use” and security overview were created to address questions and concerns (see supplemental material and methods).
An overview of the main features of the Shariant platform is provided in Figure 1.
Figure 1.
Overview of Shariant features
Main features of Shariant include two-way sharing via an application programming interface (API); sharing of structured evidence and expertise against guidelines from the American College of Medical Genetics and Genomics and Association for Molecular Pathology (ACMG/AMP);11 discrepancy resolution; submission to international databases, including ClinVar;8 and controlled access.
Controlled access
Shariant access is currently restricted to Australian laboratories conducting clinically accredited genetic testing (i.e., compliant with the National Association of Testing Authorities) who have signed the Shariant terms of use. Access is controlled via integration with standard international tooling for access management (Keycloak), and hosting is on the Amazon Web Services Sydney node. Amendments are in progress to extend access to New Zealand laboratories, given that they also follow the National Pathology Accreditation Advisory Council accreditation guidelines.6
Automated two-way sharing
So that concerns around laboratory resourcing requirements to share variant interpretations would be addressed, Shariant was designed to integrate with the interpretation system already in use by each laboratory via an API where feasible (details provided in Table S2). Further, design allows for a two-way interaction: (1) automated submission of germline variant interpretations and associated evidence to Shariant; (2) importation of data stored in Shariant for viewing in the laboratory interpretation system. Resources were dedicated to work with commercial vendors when required and to maintain connection as laboratories change interpretation systems over time. If a laboratory’s interpretation system is unable to be connected via an API, a system-formatted export may be uploaded to the Shariant web portal, and transformation of data can be performed automatically on the Shariant end (supplemental material and methods). Additionally, a configurable system-formatted export (e.g., VCF, JSON) containing interpretations from other participating laboratories can be downloaded from the Shariant webpage for import into a laboratory’s interpretation system. That is, laboratories that incorporate an export of other laboratories’ variant interpretations from Shariant into their laboratory workflow will be able to see existing classifications along with interpretation evidence.
Sharing of structured variant-interpretation evidence
As indicated by the initial survey, the ACMG/AMP guidelines11 are the most common criteria used for germline variant interpretation in Australia. Therefore, interpretation evidence was structured around these guidelines, and the flexibility to accept additional evidence fields was incorporated (version accessed 12th May 2022 included in Table S3). A minimal set of fields, including the genome build, a variant representation (usually a coding DNA Human Genome Variation Society (HGVS) expression), condition for which a variant was interpreted, zygosity, and classification/clinical significance, was deemed mandatory. Code was designed to scan for PubMed identifiers in free text, and this information is included as structured evidence with author, title, and abstract.
Variant normalization and liftover between genome builds
Accurate aggregation and connection of variants across differing variant representations and genome builds is vital for comparison of variant interpretations between laboratories. The submitted variant is matched to a genomic coordinate in the submitted genome build (both RefSeq12 and Ensembl13 transcripts are supported) and lifted over to the alternative genome build via the ClinGen Allele Registry14 and then the National Center for Biotechnology Information (NCBI) Genome Remapping Service if the former does not work. All variants are then mapped to alleles that span both GRCh37 and GRCh38. If there are changes in the variant representation, including transcript version change, right alignment, and reference-base difference, a flag is raised, and the record is not exported until the Shariant team or submitting laboratory has validated the change (see supplemental material and methods; Figure S1).
Discrepancy identification and resolution
As part of another Australian Genomics project, clinical genetic-testing laboratories provided feedback on a process for resolving between-laboratory medically significant classification discrepancies, previously defined15 as a difference between three classification tiers: pathogenic or likely pathogenic; variant of uncertain significance (VUS); and likely benign or benign. In brief, the project was tasked with analyzing the existing variant reclassification processes across Australian laboratories to inform development of consensus recommendations (H.S.S., unpublished data). Review and discussion of several historical examples of reclassification helped identify areas for improvement. These included the need to overcome logistical barriers to data sharing and communication and the implementation of an agreed-upon process for coordinating variant reinterpretation and notification. The Shariant discrepancy-identification and -resolution functionality was designed to address technical barriers related to these aspects.
Identification of a medically significant classification discrepancy within Shariant triggers an email notification to user/s from each laboratory involved. The notification provides a link allowing comparison of structured evidence between the variant records (Figure 2), and this comparison page points to a dedicated platform that facilitates and records communication within Shariant to resolve the discrepancy. Structuring of evidence has been vital for comparison of discrepant (and concordant) variant interpretations across laboratories. Communication regarding a discrepancy can be seen by any laboratory contributing to Shariant. The implementation of the discrepancy-resolution process has evolved over time, in consultation with the laboratories, highlighting the need for flexibility to address user needs.
Figure 2.
Comparison of structured evidence between variant records
(A) Example comparison of ACMG/AMP11 criteria applied by two laboratories. Red highlighting indicates a difference in the application of ACMG/AMP codes.
(B) Example comparison of citations referenced for the same variant. Ticks indicate the referencing of a citation by a particular laboratory, and gray highlighting indicates a difference between laboratories.
Automated submission to ClinVar and other international databases
Streamlined submission to international databases, predominantly ClinVar,8 is encouraged but remains as an “opt-in” for laboratories contributing to Shariant. The submission process uses the recently released ClinVar submission API and recognizes each laboratory individually (i.e., not as a consensus Shariant classification). Shariant mandatory fields overlap with most fields required by ClinVar, with the exception of a standard-condition term (e.g., Mondo Disease Ontology [Mondo],16 Online Mendelian Inheritance in Man [OMIM],17 and Human Phenotype Ontology [HPO]18). Approximately 70% of records submitted to Shariant did not meet this requirement at the time of development; thus, a new functionality was added to match free text conditions to a Mondo ontology identifier. Resources including PanelApp Australia,19,20 the Gene Curation Coalition21 and Mondo16 were used to assist in matching of a Mondo identifier relevant at the gene level (where possible), as requested by the participating laboratories (see supplemental material and methods; Figures S3 and S4).
Shariant national rollout
In the onboarding of Australian clinical genetic-testing laboratories, initial prioritization was given to public laboratories because of the nature of Australian Genomics funding. The interest of the laboratory, ease of connection to the laboratory’s interpretation system, and genetic-testing output of the laboratory were also considered.
Onboarding of laboratories required three stages.
-
(1)
Initial engagement of the laboratory, including discussion of Shariant purpose and features, laboratory willingness to participate, and distribution of Shariant documents, such as the terms of use and security overview.
-
(2)
Organizational sign-off on terms of use.
Legal sign-off by each organization (sometimes pertinent to multiple laboratories) was a significant hurdle for onboarding. Time to sign-off varied greatly, from one week to more than 1.5 years. Where possible, the connection to Shariant was developed and tested prior to the sign-off on the terms of use to minimize delays to data sharing.
-
(3)
Integration of laboratory interpretation system with Shariant.
The complexity and time involved in integration (via API connection or web portal upload) was highly dependent on the laboratory and interpretation system in use. For example, resource needs were greatly reduced during integration of a laboratory with the same commercial interpretation system as an already-connected laboratory.
Automated data transformation from a laboratory’s system-formatted export to the structured Shariant format was deemed necessary to permit regular uploads of variant interpretations and associated evidence without additional manual input (supplemental material and methods). Initial automation of this process greatly influenced time to connection, requiring laboratory-specific customization even if the same interpretation system was in use by multiple laboratories. These issues involved parsing of free text fields so that structured information not otherwise provided as a set field in the export (e.g., condition under curation) could be extracted, parsing free text so that laboratory-sensitive information (e.g., internal communication) was excluded, and mapping fields back to standard ACMG/AMP guidelines to resolve differences in interpretation schema. Automation also needed to be flexible to account for changes in the export format and interpretation schema over time.
Shariant statistics
As of December 2021, eleven laboratories across four states in Australia had shared their interpretation of largely prospectively curated variants. A total of 14,045 variant records had been shared across 2,070 genes; seven laboratories routinely submitted multiple records per variant to capture the number of patients. After multiple variant records for relevant laboratories were collapsed, there were 11,655 variants (denoted hereafter as “unique variants per laboratory”). Approximately half of these variants (53%; 6,137 variants) were classified as a VUS, 13% were designated as likely pathogenic, and 27% as pathogenic.
Variant submission schedules varied, depending on the laboratory interpretation system and resourcing, including regular (real-time, weekly, monthly) or more sporadic larger submissions (Figure 3A). It is anticipated that the frequency of variant submission will increase over time as facilitated by implementation of automated data transformation. Ingestion of Shariant data by laboratory interpretation tools generally occurs in parallel to submissions.
Figure 3.
Variants shared via Shariant over time
(A) Submission and sharing of unique variants per laboratory, presented as totals per organization over time (mm-dd-yy). Frequency of upload is dependent on the interpretation system in use by a laboratory (lab). The spike in submissions in November 2021 was due to the recent onboarding of organization 6. Organization 6 has only submitted one large batch of records, and this batch included historical data.
(B) Overall submissions of unique variants to Shariant over time and breakdown of variants submitted by one laboratory compared to variants submitted by multiple laboratories.
(C) Unique variants contributed by more than one laboratory with breakdown of comparison category: concordant-agreement variants, concordant-confidence variants, and discrepant variants. As of December 2021, concordant-agreement variants (n = 211) included those that were pathogenic, n = 145 (69%); likely pathogenic, n = 19 (9%); VUS, n = 37 (18%); likely benign, n = 9 (4%); and benign, n = 1 (0.5%). Concordant-confidence variants (n = 31) included those that were pathogenic or likely pathogenic, n = 30 (97%); and benign or likely benign, n = 1 (3%). Discrepant variants are detailed in Table 3.
It was evident that the number of variants with submissions from multiple laboratories has increased over time, consistent with more per-laboratory submissions and more new laboratories contributing to Shariant (Figures 3B and 3C). Of the 11,377 unique variants across laboratories, only 2% (260) were submitted by multiple laboratories during the time period assessed for this analysis. Of these, 232 (89%) were in complete agreement or concordant within a confidence level.
Discrepancy identification and resolution
The Shariant platform is designed to identify any medically significant classification discrepancies within or between laboratories at the time of any new upload of laboratory information. Thus, discrepancies might arise upon the first submission of a variant from a laboratory or upon resubmission of an updated classification from a laboratory. To date, these automated classification checks have identified 28 unique variants as discrepant between laboratories; this discrepancy detection rate amounts to 11% among unique variants submitted by multiple laboratories (28/260). Shariant processes have assisted resolution of 12/28 (43%) identified discrepancies (Table 3).
Table 3.
Medically significant between-laboratory classification discrepancies in order of date identified
| Gene symbol | Variant (coding HGVS) | Variant (protein HGVS) | Disease area | Classificationsa | Number of days discrepant (as of 15th December, 2021) | Resolved classification | Upgrade/downgrade | Inter-laboratory discussion required | Reason for resolution |
|---|---|---|---|---|---|---|---|---|---|
| PROC | NM_000312.3: c.565C>T | NP_000303.1: p.(Arg189Trp) | MONDO: 0005570 hematologic disorder | P vs. VUS | 6 | LP | upgrade | no | review of ACMG/AMP criteria |
| CASR | NM_001178065.1: c.1212C>T | NP_001171536.1: p.(Val404=) | MONDO: 0005066 metabolic disease; MONDO: 0005151 endocrine system disorder | VUS vs. LB | 0 | LB | downgrade | no | review of ACMG/AMP criteria |
| COL4A3 | NM_000091.4: c.4421T>C | NP_000082.2: p.(Leu1474Pro) | MONDO: 0005240 kidney disorder | VUS vs. LB | 7 | VUS | upgrade | no | review of ACMG/AMP criteria |
| ABCA4 | NM_000350.2: c. 5693G>A | NP_000341.2: p.(Arg1898His) | MONDO: 0005283 retinal disorder | VUS vs. LB | 289 | LB | downgrade | yes | new functional evidence |
| FECH | NM_001012515.2: c. 333-48T>C; NM_000140.4: c.315-48T>C | NP_001012533.1:p.?; NP_000131.2: p.? | MONDO: 0005066 metabolic disease | P vs. VUS | 106 | P | upgrade | no | reduced penetrance variant (only pathogenic if found with another loss-of-function variant) |
| TGFBR1 | NM_004612.3: c.1468A>G | NP_004603.1: p.(Lys490Glu) | MONDO: 0004995 cardiovascular disorder | LP vs. VUSA | 183 | LP | upgrade | yes | additional segregation evidence provided by one laboratory |
| POLG | NM_002693.2: c.2890C>T | NP_002684.1: p.(Arg964Cys) | MONDO: 0004069 inborn mitochondrial metabolism disorder | LP vs. VUSA | 385 | continued discrepancy | N/A | yes | no resolution—reviewed extensively by mitochondrial experts; awaiting ClinGen expert panel review |
| POLG | NM_002693.2: c.2209G>C | NP_002684.1: p.(Gly737Arg) | MONDO: 0004069 inborn mitochondrial metabolism disorder | P vs. VUS | 0 | one classification withdrawn | N/A | no | out-of-date classification uploaded |
| MYH7 | NM_000257.2: c.532G>A | NP_000248.2: p.(Gly178Arg) | MONDO: 0004995 cardiovascular disorder | LP vs. VUS | 311 | ||||
| ABCA4 | NM_000350.2: c.71G>A | NP_000341.2: p.(Arg24His) | MONDO: 0005283 retinal disorder | LP vs. VUS | 270 | ||||
| NOD2 | NM_022162.2: c.566C>T | NP_071445.1: p.(Thr189Met) | MONDO: 0005046 immune system disorder | VUS vs. LB | 249 | ||||
| DSG2 | NM_001943.3: c.3036_3037insG | NP_001934.2: p.(Tyr1013ValfsTer25) | MONDO: 0004995 cardiovascular disorder | LP vs. VUSA | 58 | VUS | downgrade | yes | additional evidence from one laboratory and ClinVar |
| F8 | NM_000132.3: c.1094A>G | NP_000123.1: p.(Tyr365Cys) | MONDO: 0005570 hematologic disorder | LP vs. VUS | 222 | ||||
| HFE | NM_000410.3: c.187C>G | NP_000401.1: p.(His63Asp) | MONDO: 0005066 metabolic disease | P vs. VUS | 222 | ||||
| NIPBL | NM_133433.3: c.1178A>G | NP_597677.2: p.(Asn393Ser) | MONDO: 0019042 multiple congenital anomalies/dysmorphic syndrome | VUS vs. LB | 57 | LB | downgrade | yes | additional evidence from one laboratory and external public source |
| CASR | NM_000388.3: c.190A>G | NP_000379.2: p.(Asn64Asp) | MONDO: 0005066 metabolic disease; MONDO: 0005151 endocrine system disorder | LP vs. VUS | 0 | LP | upgrade | no | review of ACMG/AMP criteria |
| CFTR | NM_000492.3: c.2657+2_2657+3insA | NP_000483.3: p.? | MONDO: 0005087 respiratory system disorder | P vs. VUSA | 166 | ||||
| USH2A | NM_206933.2: c.4106C>T | NP_996816.2: p.(Ser1369Leu) | MONDO: 0005283 retinal disorder | P vs. VUS | 120 | ||||
| AHI1 | NM_001134831.1: c.2988delA | NP_001128303.1: p.(Val997SerfsTer20) | MONDO: 0019042 multiple congenital anomalies/dysmorphic syndrome | LP vs. VUS | 109 | ||||
| TNFRSF13B | NM_012452.2: c.310T>C | NP_036584.1: p.(Cys104Arg) | MONDO: 0005046 immune system disorder | P vs. VUS | 109 | ||||
| KMT2D | NM_003482.3: c.12862C>T | NP_003473.3: p.(Arg4288Trp) | MONDO: 0019042 multiple congenital anomalies/dysmorphic syndrome | VUS vs. LB | 103 | ||||
| CHEK2 | NM_007194.4: c.470T>C | NP_009125.1: p.(Ile157Thr) | MONDO: 0015356 hereditary neoplastic syndrome | LP vs. VUS | 6 | one classification withdrawn | N/A | no | variant reviewed as a risk factor and therefore doesn't align with ACMG/AMP classification criteria |
| CDH1 | NM_004360.4: c.387+5G>A | NP_004351.1: p.? | MONDO: 0015356 hereditary neoplastic syndrome | VUS vs. LB | 26 | ||||
| POLE | NM_006231.4: c.2090C>G | NP_006222.2: p.(Pro697Arg) | MONDO: 0015356 hereditary neoplastic syndrome | VUS vs. LB | 26 | ||||
| MSH6 | NM_000179.2: c.3226C>T | NP_000170.1: p.(Arg1076Cys) | MONDO: 0015356 hereditary neoplastic syndrome | LP vs. VUS | 9 | LP | upgrade | no | out-of-date classification uploaded |
| POLE | NM_006231.3: c.4523G>A | NP_006222.2: p.(Arg1508His) | MONDO: 0015356 hereditary neoplastic syndrome | VUS vs. LB | 23 | ||||
| BRCA2 | NM_000059.3: c.10076A>G | NP_000050.2: p.(Glu3359Gly) | MONDO: 0015356 hereditary neoplastic syndrome | VUS vs. LB | 23 | ||||
| MEFV | NM_000243.2: c.910G>A | NP_000234.1: p.(Gly304Arg) | MONDO: 0005046 immune system disorder | VUS vs. LB | 11 |
Medically significant classification discrepancies have been defined by Harrison et al. 2017.15
Pathogenic (P), likely pathogenic (LP), variant of uncertain significance (VUS), likely benign (LB), benign (B). VUSA is a variant of uncertain significance with suspected high clinical significance.
Eight discrepancies were resolved without inter-laboratory discussion, mainly utilizing the comparison of structured evidence to review ACMG/AMP criteria, in an average of 17 days (range: 0 to 106 days, median 6 days). Four discrepancies required inter-laboratory discussion in consideration of additional evidence from one laboratory, increasing the average number of days to resolution to 147 (range: 57 to 289, median 121 days). Most (8/10) of the resolved discrepancies resulted in reclassification to a category of more certainty.
One variant has been discussed extensively but could not be resolved via the resolution process (continued discrepancy). Of the remaining 15 discrepancies awaiting resolution, five were identified from recent submissions (<30 days in discrepancy), whereas the others are longstanding (range: 103–311 days). These observations have recently led to the establishment of a formal process utilizing existing clinical-laboratory-directed multidisciplinary team meetings to address unresolved Shariant-detected discrepancies in variant classification. It should be noted that multiple discrepancies involved variants that could be considered reduced-penetrance or risk alleles, an observation reported previously.3
The overall initial discrepancy rate observed here is somewhat lower than what has been previously reported (17%–22%),3,4,22 perhaps reflecting that most variants shared have been identified prospectively, for which curations should use the most up-to-date information from the public domain (e.g., ACMG/AMP guidelines). It might also reflect differences between studies in the relative proportions of variants assigned to classification tiers at baseline. For example, the majority of medically significant discrepancies reported by Mighton et al.4 involved classification tiers likely benign or benign versus VUS, and 60% of the variants in this dataset were likely benign or benign at baseline; in comparison, only 7% of Shariant submissions were likely benign or benign at baseline. We would also hypothesize that, particularly for laboratories that routinely ingest data from Shariant, discrepancies might be minimized because evidence from other laboratories will now be visible at the time of curation.
Variant reclassification
In addition to discrepancy resolution, laboratories internally reclassified variants over the course of their involvement in Shariant. If we consider only unique variants per laboratory, 102 variants were internally reclassified over two years (Figure 4). This equates to 1.3% of total variants from the 10 laboratories with repeated submissions and is comparable to the proportion reported for ClinVar over four years (2.1%)23 and lower than that observed for studies where re-evaluation spanned a longer timeframe.24,25 Lower rates of internal reclassification than in previous studies24,25,26 could be attributed to a combination of factors, including that most variants were prospectively evaluated in the past two years according to the ACMG/AMP guidelines and with access to publicly accessible population databases (e.g., gnomAD).
Figure 4.
Reclassification of variants in Shariant
Number of unique variants that were reclassified from August 2019 to December 2021. Initial classification is represented on the x axis, and the number of each pathogenicity for reclassified variants is displayed via the colored bars. Most variants that changed classification were initially VUS (n = 73; 72%), and the majority of these resolved to likely pathogenic or pathogenic (n = 54, 74% of this subgroup).
In agreement with Harrison et al.,23 most reclassifications (91; 89%) moved to a category of more certainty (Figure 4). These included 72 variant upgrades: VUS to likely pathogenic (51, 50%); likely pathogenic to pathogenic (18, 18%); and VUS to pathogenic (3, 3%). There were 19 downgrades from VUS, all to likely benign (19%). Reclassifications to VUS included four variants that were originally likely benign (4%) and five that were originally likely pathogenic (5%). This contrasts with previous studies, where most reclassifications over time were downgrades from VUS to likely benign or benign.23,24,25,26
Use of Shariant data to study nationwide impact of new recommendations and evidence
All laboratories submitting to Shariant include structured evidence that is mapped against the original ACMG/AMP guidelines.11 These interpretations represent a valuable resource allowing for a nationwide approach to examining the impact of new recommendations for using ACMG/AMP guidelines and prioritizing additional data collation.
Using a points-based approach27 and modifying PM2 to PM2_Supporting as per recent ClinGen Sequence Variant Interpretation Working Group recommendations28 (see supplemental material and methods), the classification of 579 unique variants per laboratory applying PM2 (15%) would be changed: 219 (38%) would undergo a medically significant change (likely pathogenic to VUS), 52 (9%) variants would change from VUS to likely benign, and 308 (53%) variants would change confidence from pathogenic to likely pathogenic. This analysis demonstrates the extent to which application of the recommended revised weight to historical records might impact re-reporting, counseling of patients, and ultimately, changes in clinical management.
Functional assay evidence is only assigned a weight for approximately 10% of unique variants per laboratory. If these records are excluded, genes that had the most VUSs in the hereditary cancer context included BRCA2, PALB2, and MSH6 (Figure 5A). Across all other diseases, genes with the most VUSs included PKD1, TTN, and USH2A (Figure 5B). Although multiple factors (e.g., length of gene, type of gene) might be contributing to these high VUS numbers, the data provide a snapshot of the minimum number of families for which variant reclassification could be assisted by the incorporation of functional assay data. Such data could be used as a means of prioritizing future functional assays and/or adapting Shariant to include look-ups to large-scale functional datasets such as MAVEdb.29
Figure 5.
Genes with the most variants of uncertain significance across the context of hereditary cancer versus other diseases
Results are shown for hereditary cancer (A) and other diseases (B). Variant records that were not matched to a variant or had no ACMG/AMP11 criteria assigned were removed, and only the most recently curated record was included if more than one variant record for a variant had been submitted by the same laboratory. All records that had a strength assigned for BS3 or PS3 were also excluded. Results for the hereditary cancer genes were driven by one laboratory.
Contribution internationally
The new ClinVar submission API is now being used for Shariant-assisted automated submission. The initial submission of 385 variants from one laboratory more than doubled the number of contributions to ClinVar from Australian clinical genetic-testing laboratories, as compared to contributions prior to project initiation. On the basis of all Shariant data as of December 2021, we estimated that future Australian laboratory submissions to ClinVar would add variant interpretation evidence for over 11,000 unique variants, of which 4,000 would be novel. Moreover, this will enable routine deposition of variant data to ClinVar, as evidenced by the increased number of variant-classification submissions to ClinVar since manuscript submission; 2,146 submissions from nine laboratories as of August 2022.
Comparison to previous studies
Compared to previous national sharing efforts,4,30 Shariant has implemented several additional functionalities. These include the ability to automate transformation of data from the laboratories (as opposed to manual manipulation by the laboratory or a database coordinator prior to each upload); system integration between the laboratory interpretation system and a central database (i.e. Shariant) via an API; ability to accept multiple genome builds and facilitate liftover; and a process for identifying classification discrepancies and documenting their resolution.
Future directions
Shariant development priorities are driven collaboratively by the Shariant User Group, which meets monthly and includes all participating laboratories and the Shariant team. Resolution of variants classified as a VUS by multiple laboratories has been flagged as a priority by the User Group and has the potential to be expanded to the resolution of confidence differences. The current implementation of Shariant has focused on sharing germline variant interpretations. However, once we hold further consultations aimed at extending its design and capabilities and implement resultant modifications, we anticipate that Shariant will additionally capture and share somatic variant interpretations in the future.
Conclusion
Implementation of an automated sharing process has facilitated repeated uploads of variant interpretations and has the potential to enable widespread national real-time sharing. Sharing of structured evidence has allowed for the national comparison of interpretation processes and can be used to identify areas for future standardization. By offering a custom solution, Shariant maintains flexibility and agility. Issues around timelines to uptake have been largely governance related.
We have demonstrated that co-design with clinical genetic-testing laboratories is vital to the development and implementation of a national effort to share variant interpretations, and we provide insight into approaches that might be adapted by other national initiatives. However, it should be expected that resources are required for maintaining operations, adapting to new or updated laboratory interpretation systems, and incorporating changes in laboratory-specific or general classification guidelines. Shariant, like any other national sharing project, requires long-term and eventually sustainable funding.
Consortia
Shariant Consortium members: Lauren Akesson, Richard Allcock, Katie Ashton, Damon A. Bell, Anna Brown, Michael Buckley, John R. Burnett, Linda Burrows, Alicia Byrne, Eva Chan, Corrina Cliffe, Roderick Clifton-Bligh, Susan Dooley, Miriam Fanjul Fernandez, Elizabeth Farnsworth, Thuong Ha, Denae Henry, Duncan Holds, Katherine Holman, Matilda Jackson, Sinlay Kang, Catherine Luxford, Sam McManus, Rachael Mehrtens, Cliff Meldrum, David Mossman, Sarah-Jane Pantaleo, Dean Phelan, Electra Pontikinas, Anja Ravine, Tony Roscioli, Rodney Scott, Keryn Simons, and Oliver Vanwageningen.
Acknowledgments
The authors would like to thank Penny Glenn, who led the development of the Shariant terms of use and multiple rounds of minor amendments in consultation with laboratory legal teams. We also thank the laboratories who participated in the landscape analysis in 2017 and all individuals who attended the Human Genetics Society of Australasia conference workshop. We thank John Rowell and Valentine Hyland for their prior contributions to engagement of Pathology Queensland and the GenoVic team for efforts relating to initial integration of GenoVic laboratories. We wish to acknowledge Agilent Technologies, for its contribution of technical resources and ongoing support to achieve interoperability of Shariant with Agilent’s variant tertiary analysis system. Shariant Australia (including funding for E.T. and J.A.) is supported by Australian Genomics (NHMRC grants GNT1113531 and GNT2000001). A.B.S was supported by NHMRC fellowship funding (APP1061779, APP177524). The research conducted at the Murdoch Children's Research Institute was supported by the Victorian Government's Operational Infrastructure Support Program. The Chair in Genomic Medicine awarded to J.C. is generously supported by The Royal Children's Hospital Foundation. Hosting of Shariant on Amazon Web Services between October 2018 and September 2019 was supported by an Amazon Web Services Cloud Credit for Research grant.
Declaration of interests
The authors declare no competing interests.
Footnotes
Supplemental information can be found online at https://doi.org/10.1016/j.ajhg.2022.10.006.
Web resources
Amazon Web Services, https://aws.amazon.com/
ClinVar Submission API, https://www.ncbi.nlm.nih.gov/clinvar/docs/api_http/
Keycloak, https://www.keycloak.org/
NCBI Genome Remapping Service, www.ncbi.nlm.nih.gov/genome/tools/remap
Shariant evidence keys, https://shariant.org.au/classification/evidence_keys
VariantGrid, https://github.com/SACGF/variantgrid
Supplemental information
Data and code availability
The Shariant (VariantGrid) code generated during this study (J.A. and D.M.L.) is freely available for research use under business source license 1.1 on GitHub (https://github.com/SACGF/variantgrid). The datasets supporting the current study have not been deposited in a public repository as a result of restrictions on the sharing of non-aggregate data outlined in the Shariant terms of use. Variant interpretations and associated evidence will be deposited in ClinVar (https://www.ncbi.nlm.nih.gov/clinvar/) at submitter request.
References
- 1.Wright C.F., Ware J.S., Lucassen A.M., Hall A., Middleton A., Rahman N., Ellard S., Firth H.V. Genomic variant sharing: a position statement. Wellcome Open Res. 2019;4:22. doi: 10.12688/wellcomeopenres.15090.2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Raza S., Hall A. Genomic medicine and data sharing. Br. Med. Bull. 2017;123:35–45. doi: 10.1093/bmb/ldx024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Harrison S.M., Dolinksy J.S., Chen W., Collins C.D., Das S., Deignan J.L., Garber K.B., Garcia J., Jarinova O., Knight Johnson A.E., et al. Scaling resolution of variant classification differences in ClinVar between 41 clinical laboratories through an outlier approach. Hum. Mutat. 2018;39:1641–1649. doi: 10.1002/humu.23643. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mighton C., Smith A.C., Mayers J., Tomaszewski R., Taylor S., Hume S., Agatep R., Spriggs E., Feilotter H.E., Semenuk L., et al. Data sharing to improve concordance in variant interpretation across laboratories: results from the Canadian Open Genetics Repository. J. Med. Genet. 2021;59:571–578. doi: 10.1136/jmedgenet-2021-107738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.ACMG Board Of Directors Laboratory and clinical genomic data sharing is crucial to improving genetic health care: a position statement of the American College of Medical Genetics and Genomics. Genet. Med. 2017;19:721–722. doi: 10.1038/gim.2016.196. [DOI] [PubMed] [Google Scholar]
- 6.National Pathology Accreditation Advisory Council . First edition. 2017. Requirements for Human Medical Genome Testing Utilising Massively Parallel Sequencing Technologies. [Google Scholar]
- 7.The Royal College of Pathologists of Australasia . 2014. Massively Parallel Sequencing Implementation Guidelines. [Google Scholar]
- 8.Landrum M.J., Lee J.M., Benson M., Brown G.R., Chao C., Chitipiralla S., Gu B., Hart J., Hoffman D., Jang W., et al. ClinVar: improving access to variant interpretations and supporting evidence. Nucleic Acids Res. 2018;46:D1062–D1067. doi: 10.1093/nar/gkx1153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.The Royal College of Pathologists of Australasia . 2019. Australian Health Genetics/Genomics Survey 2017. Report of Key Findings to: Department of Health. [Google Scholar]
- 10.Stark Z., Boughtwood T., Phillips P., Christodoulou J., Hansen D.P., Braithwaite J., Newson A.J., Gaff C.L., Sinclair A.H., North K.N. Australian Genomics: a federated model for integrating genomics into healthcare. Am. J. Hum. Genet. 2019;105:7–14. doi: 10.1016/j.ajhg.2019.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Richards S., Aziz N., Bale S., Bick D., Das S., Gastier-Foster J., Grody W.W., Hegde M., Lyon E., Spector E., et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet. Med. 2015;17:405–424. doi: 10.1038/gim.2015.30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.O'Leary N.A., Wright M.W., Brister J.R., Ciufo S., Haddad D., McVeigh R., Rajput B., Robbertse B., Smith-White B., Ako-Adjei D., et al. Reference sequence (RefSeq) database at NCBI: current status, taxonomic expansion, and functional annotation. Nucleic Acids Res. 2016;44:D733–D745. doi: 10.1093/nar/gkv1189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Howe K.L., Achuthan P., Allen J., Allen J., Alvarez-Jarreta J., Amode M.R., Armean I.M., Azov A.G., Bennett R., Bhai J., et al. Ensembl 2021. Nucleic Acids Res. 2021;49:D884–D891. doi: 10.1093/nar/gkaa942. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Pawliczek P., Patel R.Y., Ashmore L.R., Jackson A.R., Bizon C., Nelson T., Powell B., Freimuth R.R., Strande N., Shah N., et al. ClinGen Allele Registry links information about genetic variants. Hum. Mutat. 2018;39:1690–1701. doi: 10.1002/humu.23637. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Harrison S.M., Dolinsky J.S., Knight Johnson A.E., Pesaran T., Azzariti D.R., Bale S., Chao E.C., Das S., Vincent L., Rehm H.L. Clinical laboratories collaborate to resolve differences in variant interpretations submitted to ClinVar. Genet. Med. 2017;19:1096–1104. doi: 10.1038/gim.2017.14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Mungall C.J., McMurry J.A., Köhler S., Balhoff J.P., Borromeo C., Brush M., Carbon S., Conlin T., Dunn N., Engelstad M., et al. The monarch initiative: an integrative data and analytic platform connecting phenotypes to genotypes across species. Nucleic Acids Res. 2017;45:D712–D722. doi: 10.1093/nar/gkw1128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.McKusick V.A. JHU Press; 1998. Mendelian Inheritance in Man: A Catalog of Human Genes and Genetic Disorders. [Google Scholar]
- 18.Köhler S., Gargano M., Matentzoglu N., Carmody L.C., Lewis-Smith D., Vasilevsky N.A., Danis D., Balagura G., Baynam G., Brower A.M., et al. The human phenotype ontology in 2021. Nucleic Acids Res. 2021;49:D1207–D1217. doi: 10.1093/nar/gkaa1043. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Stark Z., Foulger R.E., Williams E., Thompson B.A., Patel C., Lunke S., Snow C., Leong I.U.S., Puzriakova A., Daugherty L.C., et al. Scaling national and international improvement in virtual gene panel curation via a collaborative approach to discordance resolution. Am. J. Hum. Genet. 2021;108:1551–1557. doi: 10.1016/j.ajhg.2021.06.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Martin A.R., Williams E., Foulger R.E., Leigh S., Daugherty L.C., Niblock O., Leong I.U.S., Smith K.R., Gerasimenko O., Haraldsdottir E., et al. PanelApp crowdsources expert knowledge to establish consensus diagnostic gene panels. Nat. Genet. 2019;51:1560–1565. doi: 10.1038/s41588-019-0528-2. [DOI] [PubMed] [Google Scholar]
- 21.DiStefano M.T., Goehringer S., Babb L., Alkuraya F.S., Amberger J., Amin M., Austin-Tse C., Balzotti M., Berg J.S., Birney E., et al. The gene curation coalition: a global effort to harmonize gene-disease evidence resources. Genet. Med. 2022;24:1732–1742. doi: 10.1016/j.gim.2022.04.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Furqan A., Arscott P., Girolami F., Cirino A.L., Michels M., Day S.M., Olivotto I., Ho C.Y., Ashley E., Green E.M., et al. Care in specialized centers and data sharing increase agreement in hypertrophic cardiomyopathy genetic test interpretation. Circ. Cardiovasc. Genet. 2017;10:e001700. doi: 10.1161/CIRCGENETICS.116.001700. [DOI] [PubMed] [Google Scholar]
- 23.Harrison S.M., Rehm H.L. Is 'likely pathogenic' really 90% likely? Reclassification data in ClinVar. Genome Med. 2019;11:72. doi: 10.1186/s13073-019-0688-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Mighton C., Charames G.S., Wang M., Zakoor K.R., Wong A., Shickh S., Watkins N., Lebo M.S., Bombard Y., Lerner-Ellis J. Variant classification changes over time in BRCA1 and BRCA2. Genet. Med. 2019;21:2248–2254. doi: 10.1038/s41436-019-0493-2. [DOI] [PubMed] [Google Scholar]
- 25.Slavin T.P., Van Tongeren L.R., Behrendt C.E., Solomon I., Rybak C., Nehoray B., Kuzmich L., Niell-Swiller M., Blazer K.R., Tao S., et al. Prospective study of cancer genetic variants: variation in rate of reclassification by ancestry. J. Natl. Cancer Inst. 2018;110:1059–1066. doi: 10.1093/jnci/djy027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Kast K., Wimberger P., Arnold N. Changes in classification of genetic variants in BRCA1 and BRCA2. Arch. Gynecol. Obstet. 2018;297:279–280. doi: 10.1007/s00404-017-4631-2. [DOI] [PubMed] [Google Scholar]
- 27.Tavtigian S.V., Harrison S.M., Boucher K.M., Biesecker L.G. Fitting a naturally scaled point system to the ACMG/AMP variant classification guidelines. Hum. Mutat. 2020;41:1734–1737. doi: 10.1002/humu.24088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.ClinGen Sequence Variant Interpretation Working Group . 2020. ClinGen Sequence Variant Interpretation Recommendation for PM2 - Version 1.0. [Google Scholar]
- 29.Esposito D., Weile J., Shendure J., Starita L.M., Papenfuss A.T., Roth F.P., Fowler D.M., Rubin A.F. MaveDB: an open-source platform to distribute and interpret data from multiplexed assays of variant effect. Genome Biol. 2019;20:223. doi: 10.1186/s13059-019-1845-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Fokkema I.F.A.C., van der Velde K.J., Slofstra M.K., Ruivenkamp C.A.L., Vogel M.J., Pfundt R., Blok M.J., Lekanne Deprez R.H., Waisfisz Q., Abbott K.M., et al. Dutch genome diagnostic laboratories accelerated and improved variant interpretation and increased accuracy by sharing data. Hum. Mutat. 2019;40:2230–2238. doi: 10.1002/humu.23896. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The Shariant (VariantGrid) code generated during this study (J.A. and D.M.L.) is freely available for research use under business source license 1.1 on GitHub (https://github.com/SACGF/variantgrid). The datasets supporting the current study have not been deposited in a public repository as a result of restrictions on the sharing of non-aggregate data outlined in the Shariant terms of use. Variant interpretations and associated evidence will be deposited in ClinVar (https://www.ncbi.nlm.nih.gov/clinvar/) at submitter request.