Variants of uncertain clinical significance in hereditary breast and ovarian cancer genes: best practices in functional analysis for clinical annotation

Abstract

Germline DNA tests to identify pathogenic variants in genes linked to hereditary breast and ovarian cancer susceptibility have become widely available. However, the clinical utility of genetic testing depends on reliable evidence-based classification of sequence variants. Determination of pathogenicity traditionally relies on painstaking pedigree-based segregation analyses. However, the rapid increase in usage of germline DNA tests has led to the discovery of a large number of variants of uncertain clinical significance (VUS). For most VUS there is insufficient information for segregation analysis and therefore assessment of functional consequences is increasingly being used to support clinical annotation. Functional assays need to be accurate, robust, and reproducible to be used for clinical purposes. Here we use the lessons learned from BRCA1 and BRCA2 to identify best practices for the use of functional assays for clinical annotation of germline VUS in breast and ovarian cancer genes. We provide recommendations for the interpretation and use of established functional assays as well as for the development of new assays.

Introduction

Since the identification and cloning of BRCA1 in 1994 [1], and shortly thereafter of BRCA2 [2], genetic tests of germline DNA to identify pathogenic variants in genes linked to hereditary breast and ovarian cancer (HBOC) have become mainstream [3]. These tests are critical to identify women at increased risk relative to the general population. Women at moderate risk (2 ≤ Relative Risk (RR) < 4) may benefit from enhanced screening and chemoprevention while those at high risk (RR ≥ 4), including those with BRCA1 and BRCA2 pathogenic variants, may also benefit from preventive surgery. Germline mutation testing is also becoming increasingly relevant in the cancer treatment setting because carriers of pathogenic variants in BRCA1/2 may benefit from PARP (Poly-ADP Ribosyl Polymerase) inhibitors [4 5]. Importantly, genetic tests can identify individuals in HBOC families who do not carry the relevant predisposing allele and are not at elevated risk of cancer [6].

A significant fraction of documented variants in BRCA1 and BRCA2 are considered variants of uncertain clinical significance (VUS), for which cancer association has not been assessed or could not be determined due to insufficient information (Table 1). In ClinVar (https://www.ncbi.nlm.nih.gov/clinvar/), a clinically-oriented database, currently ~37% of BRCA1 and ~45% of BRCA2 unique variants recorded are VUS. Thus, there is a critical need to classify variants according to their pathogenicity.

Table 1.

Fraction of VUS in BRCA1 and BRCA2

Databases:	BICa		ClinVarb		BRCA Exchangec		gnomADd
	n	% (%VUS)	n	% (%VUS)	n	% (%VUS)	n	%
BRCA1 Unique variants	1781	100	5821	100	7898	100	2936	100
BRCA1 VUS	891	50.0 (100)	2146	36.9 (100)	5186	65.7 (100)	n/a	n/a
BRCA1 Missense	607	34.1 (68.1)	1715	29.5 (79.9)	1892	24.0 (36.5)	938	31.9
BRCA1 Missense VUS	569	31.9 (63.9)	1633	28.1 (76.1)	1714	21.7 (33.1)	n/a	n/a
BRCA2 Unique variants	2000	100	8119	100	10422	100	4262	100
BRCA2 VUS	1065	53.3 (100)	3615	44.5 (100)	6980	67.0 (100)	n/a	n/a
BRCA2 Missense	891	44.6 (83.7)	3111	38.3 (86.1)	3484	33.4 (49.9)	1909	44.8
BRCA2 Missense VUS	838	41.9 (78.7)	3011	37.1 (83.3)	3190	30.6 (45.7)	n/a	n/a

^aBIC, Breast Cancer Information Core (https://research.nhgri.nih.gov/bic/) is a locus specific database established in 1995 for BRCA1 and BRCA2 variants, including loci primarily found in clinical or research testing

^bClinVar (https://www.ncbi.nlm.nih.gov/clinvar/) is a public archive of reports of the relationships among human variations and phenotypes and includes submissions reporting variants found in patient samples from clinical or research testing, and from the literature (note: ClinVar also includes BIC data)[69]. VUS counts also include conflicting assessments

^cBRCA Exchange (http://brcaexchange.org/) pools data on BRCA1/2 genetic variants and corresponding clinical data from around the world (including BIC, ClinVar, 1000 Genomes Project). BRCA Exchange is part of the Global Alliance for Genomics and Health. VUS counts also include ‘not yet reviewed’

^dgnomAD, The Genome Aggregation Database (http://gnomad.broadinstitute.org/), initially released as ExAC aggregates and harmonizes both exome and genome sequencing data from a wide variety of large-scale sequencing projects. It does not contain pathogenicity assessments. All searches were conducted in December 2017.

Over the past decade, functional assays have emerged that can be included as a source of evidence to classify variants according to their pathogenicity, with the potential to greatly accelerate classification [7]. Here we discuss several technical and conceptual aspects relevant for the use functional assays in the classification of variants. We present best practice recommendations to improve annotation quality and accuracy, and to provide a basis for the comparison and integration of functional data from different laboratories (Box 1). For the coming years, we anticipate that recent technological developments such as VAMP-Seq (variant abundance by massively parallel sequencing) or high-throughput CRISPR-based saturation mutagenesis will enable the functional assessment of every missense variant for all moderate and high risk HBOC genes [8–10]. Once established and validated, these catalogues of functional data will provide valuable information for clinical annotation. The recommendations proposed here are the result of a discussion that started at a Netherlands Cancer Institute (NKI) workshop on Functional Analysis of Sequence Variants in Hereditary Breast and Ovarian Cancer Genes (Amsterdam, Netherlands) and was followed by additional discussion and extensive refinement. It represents a consensus view that was self-developed by an international group of investigators (the authors) who have been active in this field.

Box 1. Summary recommendations for the development, reporting and interpretation of functional assays.

• Assess the strength of evidence for association between each gene and HBOC risk to evaluate the potential clinical utility of a proposed functional assay

• Consider the assumptions, biological characteristics, controls and limitations of each assay

• Choose genomic DNA/cDNA/Protein sequences that correspond to the coding sequence of the most commonly found haplotype in non-affected individuals to be used as a reference (wild-type)

• A minimal set of non-pathogenic and pathogenic variants should be used as internal reference for each run of an assay. Larger sets of reference variants should be used to assess the overall sensitivity and specificity of an assay

• Verify that elements of the assay (reagents and data) have been through quality control, including periodical verification of cell line and strain identity (e.g. identity by short tandem repeat analysis for mammalian cells and phenotyping for yeast strains) and quality (mycoplasma testing)

• Due to protein stability issues, exercise caution when developing and interpreting results from model organisms that are cultured at temperatures lower than 37°C

• Inspect data to identify and correct batch effects

• Do not assume that intermediate levels of activity necessarily reflects intermediate risks

• In addition to loss-of-function effects also consider dominant negative and gain-of-function effects

• When reporting results use explicitly defined terminology and aim for the development of a controlled vocabulary

• Be explicit about assay’s limitations, performance metrics and thresholds used to classify variants

Assessment of the evidence for association of each gene with HBOC risk

The first step in developing or interpreting results from functional assays is to understand the level of evidence that links a particular gene to breast and ovarian cancer risk [3]. To date, there are nine genes for which an association between protein-truncating variants and breast cancer risk has been established (ATM, BRCA1, BRCA2, CDH1, CHEK2, PALB2, PTEN, STK11, and TP53) and several more (BARD1, FANCM, NBN, NF1, MLH1, MSH2, MSH6, PMS2, RAD51C, and RAD51D) [3 11–13] for which association has been suggested but not yet firmly established. At least twelve genes have been implicated in ovarian cancer risk (ATM, BRCA1, BRCA2, BRIP1, MLH1, MSH2, MSH6, PALB2, PMS2, RAD51C, and RAD51D) [14 15].

Development of functional tests for emerging genes provides opportunities to uncover new mechanistic aspects of their biology and identify functional domains. However, developers of functional assays should consider that clinical recommendations are unlikely to be made based on variants in genes for which the association has not been robustly established. Thus, a detailed understanding of the strength of evidence for association between each gene (and its variant alleles) and HBOC risk should be sought to evaluate the clinical utility of a proposed functional assay.

An additional aspect to consider when developing a functional assay is the proportion of missense VUS that are probably pathogenic. Missense variation is unlikely to significantly affect the overall protein function when located in disordered regions or in repeat motifs. Therefore, functional assays for these regions (or for a protein with a large portion of its coding sequence composed by these regions) may not be a priority.

Assessment of variant pathogenicity

Genes implicated in HBOC are tumor suppressor genes and therefore variants leading to disruption of function(s) are usually considered pathogenic for clinical purposes. Notable exceptions of variants with dominant negative or gain-of-function have also been reported [16]. Loss-of-function genetic alterations include frameshift and nonsense variants leading to truncation of a functionally important segment of the protein, alterations of donor and acceptor splice sites, and large genomic rearrangements altering segments of the coding region. Conversely, synonymous changes without effect on mRNA splicing are considered non-pathogenic. These variants can be reliably classified by a rule-based system that incorporates general DNA/RNA/protein rules and takes into account exceptions specific to each gene (ENIGMA rules for classification of BRCA1 and BRCA2 variants: https://enigmaconsortium.org/library/general-documents/enigma-classification-criteria/).

For a significant fraction of rare variants pathogenicity cannot be predicted based on DNA changes alone. Primarily, these variants include small deletions or insertions that do not disturb the reading frame, missense changes, intronic and exonic variants that may lead to altered mRNA splicing and in-frame exon deletions or duplications. Missense variants represent the largest contributor to this class, making up to 79.9% and 86.1% of all BRCA1 and BRCA2 VUS in ClinVar, respectively (Table 1).

Classification of these BRCA1/2 variants for clinical use can be based on the ACMG/AMP (The American College of Medical Genetics and Genomics; Association for Molecular Pathology) guidelines in which pathogenicity is determined by the entire body of evidence [7]. In this proposed five-tier classification, variants with greater than 90% certainty of being pathogenic (which includes likely pathogenic and pathogenic variants), are considered actionable, and carriers are managed as high risk (RR ≥ 4). Evidence is qualitatively weighed as strong, moderate, supporting, or not used. Functional data provides strong (PS3: well established functional studies show a deleterious effect) and moderate (PM1: mutational hot spot or well-studied functional domain without benign variation) evidence for pathogenicity; and strong (BS3: well established functional studies show no deleterious effect) evidence for benign impact [7]. Reproducible and robust functional assays that have been validated are considered the most well-established source.

Alternatively, classification of BRCA1/2 variants can be based on a multifactorial statistical model that incorporates data on family history, co-segregation, and co-occurrence with another allele with a known pathogenic variant in the same gene (because biallelic inactivation is embryonic lethal while biallelic partial loss of function leads to Fanconi anemia) [17]. In the IARC proposed five-tier classification variants with greater than 95% certainty of being pathogenic are considered actionable [18]. Variants that reach greater than 1.0% allele frequency in the population are considered unlikely to be pathogenic on their own but there is simply insufficient clinic and family-based genetic information to determine the likelihood of pathogenicity of many uncommon (< 1.0%) variants. Currently, functional data is not integrated in these multifactorial statistical models.

The effects of these rare VUS can be predicted by a wide variety of publicly available in silico tools with variable performance [19 20]. For tools that use multiple sequence alignments, performance has been tied to the choice of alignments and calibration [21–23]. Reliance on multiple sequence alignment and evolutionary approaches may also generate false negatives. For example, Kondrashov et al. have estimated that approximately 10% of variants that are classified as “tolerated”, because a corresponding amino acid residue is found in the cognate position in another species, only score as “tolerated” because of compensatory variation elsewhere in the protein sequence [24]. Despite these limitations, algorithms are constantly improving and the concordance between some predictors and empirical data is sufficiently high (Figure 1) to guide prioritization of variants for functional assessment. However, empirical functional data will be necessary for the robust clinical annotation of uncommon variants.

Figure 1. Circos plot illustrating the concordance between AlignGVGD predictions and experimental data derived from analysis of BRCA1 C-terminal variants (aa 1396–1863).

Variants were analyzed using the transcription activation assay [29] and assigned to functional classes by VarCall [57]. Blue ribbons show that all variants scoring C0 and C15 in Align GVGD and predicted to have no or little functional impact score as non-pathogenic or likely non-pathogenic (fClass1 or 2) in a validated functional assay. Conversely, most variants scoring as C65 and predicted to have a functional score as fClass5 (thick purple ribbon). Despite the strong concordance between alignGVGD and the transcriptional assay, a small but significant fraction of variants scoring as fClass1 were incorrectly predicted (C45-C65).

The spectrum of low, moderate, and high risk alleles in HBOC genes

When developing or interpreting a functional assay for VUS classification, the level of risk conferred by pathogenic variants should be considered. Findings of pathogenic variants in a low penetrance gene (RR < 2) currently do not trigger clinical recommendations making the development of a functional assay a low priority. It is also plausible that variants within the same gene may span the spectrum of low (RR < 2), moderate (2 ≤ RR < 4), and high (RR ≥ 4) risk. In other words, distinct “pathogenic” variants in the same gene may carry significantly different levels of risk.

Currently, the multifactorial statistical model for classification of BRCA1 or BRCA2 variants determines whether a variant is likely to be pathogenic. The clinical inference of the IARC classification is based on variants that typically are associated with a high cancer risk comparable to a truncating variant in BRCA1 or BRCA2 (RR ≥ 4) [18]. However, it is now clear that some pathogenic missense variants in BRCA1 (p.R1699Q and p.V1736A) and BRCA2 (p.Y3035S and p.G2508S) confer only moderate breast cancer risk (2 ≤ RR < 4) [25–27]. On the other hand, the BRCA2 p.K3326X, classified by the model as non-pathogenic, was shown to confer a mildly increased risk (RR = 1.4) of breast and ovarian cancer [28]. Although finding this variant would not trigger a change in clinical recommendation currently, this variant can contribute to polygenic risk scores based on common genetic variants that are now being used for risk stratification, and may prove effective for selection of screening and prevention options.

Some assays may have the ability to reflect different levels of risk depending on the dynamic range of the read-out but also on the specific biological assay being performed. It is important to stress that it should not be assumed that intermediate levels of activity in a biochemical or biological assay necessarily reflects intermediate risks. Several reference variants with known intermediate risks should be used to determine the ability of an assay to reflect the continuum of risk. While the transcription activation assay for BRCA1 does not seem to discriminate between variants with intermediate risks from variants associated with high risk [29], the BRCA2 homologous recombination (HR) assay may be able to distinguish high from moderate and low/neutral as suggested by functional assessment of variant p.Y3035S [30]. For genes in which pathogenic variants are clearly associated with disease risk, a two-stage reporting system has been proposed, i.e. the first stage would establish pathogenicity of the variant based on multiple criteria and the second stage would denote the likely severity or clinical consequence for that variant (high, moderate or low risk) [31]. Capturing the full spectrum of risk associated with distinct pathogenic variants is a critical challenge for assay development and for reporting laboratories.

Classification from both ACMG/AMP and multifactorial models are designed to distinguish high risk (RR ≥ 4; actionable) from not high risk (RR < 4) variants and are currently not suitable to identify moderate risk variants. From a clinical standpoint, while these models have a binary outcome (actionable vs. non-actionable), carriers of moderate risk variants (2 ≤ RR < 4) may also benefit from enhanced screening [32].

Functional assays for HBOC gene variant classification

For the purposes of our discussion, “function” is considered as any aspect defined by the Gene Ontology Consortium [33] molecular function, cellular component, and biological processes. A “functional assay” is generally defined as any in vitro and in vivo system able to determine the impact of a variant by assessing its effect on protein stability, conformation, and function. Thus, assays for splicing alterations are not considered functional assays for the purposes of this manuscript (for assessment of splice variants please refer to refs. [34 35]).

Several characterized functions of the BRCA1 and BRCA2 proteins have been exploited in the development of functional assays (reviewed in [36–38]). Reflecting their central role in DNA damage repair, many assays revolve around measuring the ability of the variant to promote survival following DNA insults, such as treatment with ionizing radiation or DNA damaging compounds. In addition to these viability assays, specific biochemical assays such as those measuring homologous recombination or ubiquitylation, are also rooted in the known biology of BRCA1 and BRCA2. Finally, more limited biochemical assays, measuring binding to specific interacting proteins have also been applied to the functional analysis of variants [36–38].

In general, there is enough evidence to tie each of these functions to the etiology of tumors arising in carriers. However their individual contribution to cancer risk is unclear, making it difficult to determine which assay is more or less biologically appropriate, or to assign different weights to results obtained from different assays. Rather, the determination of which assays should be used for clinical annotation relies on their accuracy, and not on their biological properties. Preliminary analysis has shown that these functional assays display very high (> 80%) sensitivity and specificity (Table 2) [39].

Table 2.

Categories and performance of functional assays for BRCA1 and BRCA2

Gene	Assay	Set-up	Read-out	# of variants assessed [# of known non-pathogenic; pathogenic]a	Sensitivity (95%CI)b	Specificity (95%CI)b	Ref.
BRCA1	Colony size	cell-based (yeast)	complementation/ perturbation	40 [15; 25]	0.96 (0.80–1.00)	0.93 (0.68–1.00)	[58]
BRCA1	Yeast localization	cell-based (yeast)	complementation/ perturbation	40 [15; 25]	0.84 (0.64–0.95)	0.93 (0.68–1.00)	[58]
BRCA1	Transcription activation	cell-based (HEK293T)	reporter system	204 [25; 10]	1.00 (0.75–1.00)	1.00 (0.83–1.00)	[29]
BRCA1	BARD1 binding	cell-based (yeast)	reporter system	35	n/a	n/a	[70]
BRCA1	UbcH5a binding	cell-based (yeast)	reporter system	35	n/a	n/a	[70]
BRCA1	Uniquitin ligase activity	cell-free (in vitro)	in vitro enzymatic activity	35	n/a	n/a	[70]
BRCA1	Protease sensitivity	cell-free (in vitro)	in vitro binding activity	117 [10; 14]	0.79 (0.49–1.00)	0.80 (0.44–0.98)	[56]
BRCA1	Phosphopeptide binding activity	cell-free (in vitro)	in vitro binding activity	117 [10; 14]	0.86 (0.57–0.98)	1.00 (0.69–1.00)	[56]
BRCA1	Phosphopeptide binding specificity	cell-free (in vitro)	in vitro binding activity	117 [10; 14]	1.00 (0.77–1.00)	0.99 (0.56–1.00)	[56]
BRCA1	ES cell survival	cell-based (mouse ES cells)	complementation/ perturbation	86 [25; 9]c	n/a	n/a	[62]
BRCA1	Cisplatin sensitivity	cell-based (mouse ES cells)	complementation/ perturbation	86 [25; 9]	1.00 (0.63–1.00)	1.00 (0.83–1.00)	[62]
BRCA1	BARD1 binding	cell-based (yeast)	reporter system	1,287 [3; 19]	n/a	n/a	[71]
BRCA1	Uniquitin ligase activity	cell-free (in vitro)	in vitro enzymatic activity	1,287 [3; 19]	n/a	n/a	[71]
BRCA1	Haploid cell survival	cell-based (HAP1 cells)	complementation/ perturbation	3,893 [22; 162]d	0.967	0.982	[10]
BRCA1	Homologous recombination	cell-based (RG37-shBRCA1 cells)	complementation/ perturbation	78 [6; 7]	1.00	1.00	[72]
BRCA1	Localization	cell-based (RG37-shBRCA1 cells)	complementation/ perturbation	78 [6; 7]	0.714	1.00	[72]
BRCA1	Protein expression and stability	cell-free (in vitro)	In vitro solubility and thermostability	78 [6; 7]	0.714	0.83	[72]
BRCA1	Phosphopeptide binding activity	Cell-free (in vitro)	in vitro binding activity	42 [5; 2]	n/a	n/a	[72]
BRCA1	Homologous recombination	Cell-based (HeLa-DR-FRT)	complementation / perturbation	1,056 [5; 8]	0.875	1.00	[9]
BRCA1	Homologous recombination	Cell-based (HEK293T)	complementation / perturbation	35 [23; 5]e	1.00	1.00	[73]
BRCA2	Homologous recombination	Cell-based (V-C8 cells)	complementation / perturbation	64 [18; 13]	1.00 (0.75–1.00)	1.00 (0.82–1.00)	[30]
BRCA2	Homologous recombination	Cell-based (V-C8 cells)	complementation / perturbation	139 [12; 13]	1.00 (0.75–1.00)	1.00 (0.69–1.00)	[74]
BRCA2	Homologous recombination	Cell-based (mouse ES cells)	complementation/ perturbation	43 [20; 15]	1.00 (0.78–1.00)	1.00 (0.83–1.00)	[75]

Only assays in which more than 30 variants were tested are listed.

^aKnown pathogenic and non-pathogenic variants used for estimating sensitivity and specificity are those classified using the multifactorial model as IARC Classes 1,2,4 or 5 (Ref.[18]), unless otherwise indicated.

^bAs originally published, unless otherwise stated.

^cUsed missense variants classified by multifactorial model as IARC Classes 1,2,4 or 5 (Ref.[18]) plus the recently classified G1770V variants as pathogenic.

^dUsed ClinVar as a source of known pathogenic and non-pathogenic variants.

^eUsed as non-pathogenic variants Align-GVGD grade of C0 and IARC Class 1, and as pathogenic variants Align-GVGD grade of C35-C65 and IARC Class 5).

After the development of a large number of functional assays for high risk genes (BRCA1, BRCA2 and TP53) [36 37 40], significant attention has been focused on developing assays for other high/moderate risk genes such as PALB2, ATM, and CHEK2 [41–46]. However, here we will focus on functional assays for missense variants of BRCA1 and BRCA2 as exemplars from which we have derived general guidelines.

Functional assays can be defined by three broad categories according to their experimental set-up (cell-free or cell-based), host (human or model organism) and read-out. Result interpretation requires careful consideration of the assumptions, the biological characteristics, and limitations of each assay.

Cell-free systems either test a specific biochemical activity in vitro (e.g. phosphopeptide binding, ubiquitin ligase activity, DNA combing, DNA binding, DNA recombination), proteinprotein interactions (e.g. yeast two-hybrid screening, co-immunoprecipitations), or the effect of different factors on protein structure and stability (e.g. protease sensitivity, calorimetry). Interpretation of results from cell-free assays should consider that they are restricted to specific functions, sometimes limited to specific regions of the protein, and may be particularly sensitive to temperature, buffer conditions, and concentrations of exogenous substrates.

Cell-based systems use a human or model organism (e.g. yeast, bacteria, or mouse) host cell as the basis for the assay. Cell-based systems can be further distinguished as in cellulo (when the assay context is a single cell) or in vivo (in the context of a whole metazoan organism), although there are currently no established in vivo functional assays for VUS. We recommend periodical authentication of cell line and strain identity by short tandem repeat analysis and phenotyping, respectively. Cell lines should be checked regularly for mycoplasma infection. Interpretation of results from assays performed in model systems should consider the degree of divergence of proteins from the host involved in the assay, differences in biology, and in growth conditions.

Cell-based assays can be further defined by read-out. Reporter systems include those in which the read-out for functional impact is an ectopic reporter (e.g. transcription activation or HR assays) or in which ectopic overexpression in a heterologous system leads to a defined phenotype (e.g. small colony phenotype in yeast). Limitations of reporter systems based on ectopic expression may include artifacts of over expression. Alternatively, assays in which the full length variant allele/protein replaces the endogenous gene/protein and defined biological processes are assessed are considered complementation/perturbation assays. Some assays may combine reporter systems and complementation.

Ultimately, the value of an assay will depend on its performance, defined using a set of known non-pathogenic and pathogenic control variants as reference (see below). Given the complexity of the interaction of multiple biochemical functions and breast and ovarian cancer phenotype it is uncertain that there will be a single comprehensive and highly accurate functional assay. Rather, the combination of approaches using diverse sources of data obtained with transparent methodology and careful interpretation is likely to solve the challenges of VUS in HBOC genes.

Mouse models, although not suitable for high throughput analysis, can be helpful in determining the effect of such variants on tumor predisposition and treatment response. Brca1 mutant mice expressing the p.I26A variant showed that the E3 ubiquitin ligase activity of BRCA1 is dispensable for tumor suppression and mice expressing 185delAG (c.68_69delAG; p.E23VfsTer17) revealed the hypomorphic nature of this pathogenic variant in response to therapy [47–49]. A knock-in mouse model of the BRCA2 p.G25R variant, which had no effect on ES cell viability but had subtle defect in HR, showed a significant increase in tumor formation in mutant animals [50]. Similarly, the effect on tumor predisposition of an alternatively spliced Brca2 transcript lacking exons 4–7 was revealed in mutant mice lacking these exons [51].

Requirements for a clinically relevant functional assay

The analytical validity is the degree of accuracy with which a functional assay correctly classifies variants as pathogenic or non-pathogenic. For each assay performance metrics (True Positive Rate or sensitivity; True Negative rate or specificity; False Positive and False Negative Rates; Positive and Negative Likelihood Ratios; False Discovery Rate; False Omission Rate; Positive Predicted Value or precision; Negative Predictive Value; Accuracy; and Diagnostic Odds Ratio) should be derived from testing a panel of known pathogenic and non-pathogenic variants. The recommendation is to chose a set of pathogenic and non-pathogenic missense controls whose likelihood of pathogenicity has been established by the multifactorial statistical model and can be found in a recent ENIGMA publication [36 37 52]..

There are no specific recommendations about which threshold of sensitivity or specificity should be used to consider the inclusion of data from a functional assay for variant classification. Plon and colleagues [18] have pointed out that clinical decisions based on predictive values of 80–85% are normally used in oncology. A more strict approach would require that the lower bound of the 95% confidence interval be above the suggested 80–85% threshold but that may be difficult to achieve for genes for which there are very few known pathogenic and non-pathogenic variants to use in a validation set, which will be reflected as wider 95% intervals.

Controls are critical for validation of assays, assessment of dynamic range, and to determine metrics of performance such as sensitivity and specificity. Some thought should be given to decide on genomic DNA/cDNA/Protein sequence that corresponds to the coding sequence in the most commonly found haplotype in non-affected individuals to be used as a reference (wild-type). Note that differences in frequency of common alleles may exist across different populations. This reference cDNA or genomic sequence must be included in every experiment. Variants are scored as having functional impact or not depending on how much they differ from the reference.

It is recommended that within each run of the assay, in addition to the reference sequence, at least one known missense pathogenic and one missense non-pathogenic variant is included. If possible, known missense variants for each protein domain are recommended. To account for the range of variation of non-pathogenic variants, additional known non-pathogenic variants should also be included. Addition of hypomorphic (attenuated) variants with established intermediate risk may help calibration of assay results. Concerning VUS in genes for which there are no known missense non-pathogenic and pathogenic variants, alternative approaches, such as the use of missense variants with greater than 1.0% allele frequency and truncating variants as benign and pathogenic controls, respectively, might provide a yardstick. Results from assays using only truncating variants as pathogenic control should be interpreted with caution as truncating variants may not produce detectable protein, with implications to measuring baseline activity.

Lessons from BRCA1 and BRCA2 functional assessment

BRCA1 and BRCA2 have 1,863 and 3,418 codons, respectively. If we consider all possible single nucleotide changes in these codons, 11,015 and 20,169 unique missense variants are generated (some changes will result in the same amino acid changes), respectively. Because many have never or only sparsely been observed, we expect that most are rare (< 0.01%) such that data from functional assays will be required to assess their likelihood of pathogenicity. In order to maximize the chances of identifying pathogenic variants, investigators have focused on functional domains and motifs in which it seems more likely that variants affect protein function. Thus far, most assays have focused on variants at the RING and BRCT domains of BRCA1 and at the DSS1 and DNA interaction domain of BRCA2 [9 10 36 37]. Several functional assays have been described for BRCA1 and BRCA2, but few have tested large (> 30) sets of variants (Table 2).

For specificity and sensitivity calculations, variants are classified according to a binary classification based on the functional data: functional impact versus no functional impact. Variants with intermediate scores are ignored. This classification is then compared to a binary classification of a reference panel which combines the non-actionable Classes 1 and 2 (benign and likely benign) or actionable Classes 4 & 5 (likely pathogenic and pathogenic). This simplification allows for the estimation of the assay’s ability to correctly identify actionable and non-actionable variants. Most published functional assays have reported high sensitivity and specificity, often close to 100% (Table 2). However, these numbers partially reflect the relatively low numbers of known variants used to assess specificity and sensitivity. To obtain a better sense of an assays performance, it is critical to record and report the lower bounds of the 95% confidence interval.

Several assays have been developed using yeast (Saccharomyces cerevisiae) which provides a cost-effective and practical platform to evaluate missense variants. However, caution is warranted when interpreting results from model organisms that are cultured at temperatures lower than 37°C. Some pathogenic variants are relatively stable at lower temperatures (30°C versus 37°C) and may score as false negatives [53–55], reflected in the assay’s slightly lower sensitivity (Table 2).

It is important to note that contradictory results, for example a variant scoring pathogenic in a functional assay while being classified by clinical and family data as non-pathogenic, provide opportunities for discovery. BRCA1 variant p.V1736A scored as pathogenic in several functional assays and by in silico prediction tools [56] despite being classified as non-pathogenic due to a co-occurrence with the known pathogenic p.D821Ifs*25 variant in the same patient. However, upon further examination it was found that the carrier presented several features (e.g. developmental delay, microcephaly, short stature, very early onset ovarian cancer) pointing at hypomorphic BRCA1 activity [25]. Detailed genetic and functional investigation of the p.V1736A variant led to the discovery of the first documented carrier of biallelic pathogenic variants in BRCA1. This analysis established the existence of variants with intermediate effects and highlighted the power of functional assessment [25].

Although there are several missense variants that have displayed intermediate effects in vitro or in mouse models, only three variants, in addition to p.V1736A, have been established as hypomorphic in humans [25 27]. BRCA1 variant p.R1699Q (c.5096G>A; OR = 4.29) and BRCA2 variants p.Y3035S (c.9104A>C; OR = 2.52) and p.G2508S (c.7522G>A; OR = 2.68), have been shown to confer moderate increased risks. There is currently no consensus guidelines about their clinical management but the ENIGMA consortium has recommended breast surveillance for female carriers of p.R1699Q based on mammogram annually from age 40 and bilateral salpingo-oophorectomy should be considered based on family history [32]. Care should be exercised in the choice of statistical treatment of the data generated in functional assays. Results from assays are usually normalized using the mean of the activity of the wild-type or reference sequence. Normalization allows for comparisons across multiple experiments and, in some cases across multiple assays since variant activity is being represented as a percentage of the wild-type activity. Batch effects may be problematic (variance of the wild-type activity across multiple batches should also be taken into account) and some statistical models have taken that in consideration [57].

A more difficult task is the decision of a specific threshold of activity to separate pathogenic from non-pathogenic variants. Several approaches use arbitrary thresholds (e.g. 20% or 50% of wild-type activity; number of standard deviations from the wild-type reference; highest activity of a pathogenic variant and lowest activity of a non-pathogenic variant) or linear regression. Recent approaches have moved towards providing a probabilistic interpretation (i.e. likelihood of pathogenicity of the variant given the functional data) [57 58].

Probabilistic approaches also provide a path for integration of functional data with other data sources used to classify variants. By generating likelihood ratios (LRs) from the raw or processed functional readouts (e.g. viability counts, luciferase activity, GFP intensity), these approaches allow for the incorporation of functional assays as a data source into traditional multifactorial models that have so far not integrated functional data [57].

Integration of functional data can also be achieved using the ACMG/AMP classification model. According to the ACMG/AMP, ‘well-established assays’ can be used to obtain a PS3 or BS3 (strong evidence) criterion but there are no specific guidelines, which are likely to be established by expert panels for each gene. For example, concordant results from at least three independent validated assays would be needed for PS3 or BS3 (strong) criteria; while concordant results in two independent validated assays would generate a PM1 (moderate) criterion.

Functional assays based on sensitivity to therapeutic compounds

Insight in the importance of BRCA1 and BRCA2 for HR led to the development of carrier-specific treatment modalities for breast and ovarian cancers [59–61], in particular with the use of poly ADP ribose polymerase inhibitors (PARPi) which are synthetic lethal with BRCA1 or BRCA2 deficiency. The therapeutic window for these types of treatment is greatly increased by the fact that BRCA1 and BRCA2 mutation carriers are generally heterozygous for the pathogenic allele, while tumor cells frequently undergo loss of the wild-type allele. Thus, while the tumor cells are not viable in the presence of PARPi, non-tumor cells survive, making the therapy highly effective yet well tolerated.

Because inactivation of the gene product is required both for disruption of HR and sensitivity to PARPi or platinum compounds, functional assays based on sensitivity to these compounds can be used as an indirect read-out for HR to classify germline variants according to their pathogenicity [62].

In addition to the classification of germline variants according to associated cancer risk there is an emerging need to classify germline and somatic variants according to their response to PARPi or DNA damaging compounds to predict drug response. PARPi have been approved in the US and Europe for treatment of advanced and metastatic breast and ovarian cancers. It is unclear the extent to which sensitivity to PARPi or platinum compounds measured in a functional assay predicts in vivo tumor sensitivity. Pathogenic variants conferring high cancer risk with hypomorphic activity towards PARPi response are known to exist. For example, mouse tumor cells carrying the pathogenic Brca1 p.C61G variant showed a poor response to platinum compounds and to PARPi, and resistance rapidly emerged [63]. Importantly, determining whether a BRCA1 or BRCA2 variant found in tumor tissue confers sensitivity to a given drug may need further clinical information to calibrate the functional assays for this purpose.

Many other genes implicated in HBOC (such as ATM, CHEK2, PALB2, and TP53) are involved in the DNA damage response, suggesting that associated tumors may also have a targetable DNA repair defect. Importantly, recent results from the NOVA, Study 19, and ARIEL3 studies have raised the possibility of a significant benefit of PARP inhibitors in ovarian cancer irrespective of BRCA mutation status [64–66].

Future challenges and opportunities

As we move forward, functional assays should be able to face a significant increase in the number of genes and variants to be analyzed for clinical use. The number of alleles for genes predisposing to breast and ovarian cancer is expected to be very large. BRCA1 and BRCA2 have 7,898 and 10,422 unique alleles documented (Table 1; BRCA Exchange: http://brcaexchange.org/) and other HBOC genes are expected to significantly add to the number of predisposing alleles. In addition, recent systematic germline sequencing efforts of breast and ovarian cancer cases are expected to reveal additional genes and variants associated with risk.

To face the rapid increase in variants, high throughput functional assays that can generate and analyze large numbers of variants have been developed for BRCA1 [9 10]. Initially these consisted of the large scale generation and analysis of ectopically expressed cDNA constructs. Recent development of CRISPR-assisted gene targeting has now also allowed saturated mutagenesis at endogenous loci, exemplified by high-troughput mutagenesis of the BRCA1 RING and BRCT domains in haploid cells [9 10]. Likewise, assays based on human primary cells from carriers may also be used to capture more subtle functional defects that depend on the carrier’s genetic background, but may be less amenable to high throughput approaches [42]. Such high-throughput approaches can provide catalogues of potentially pathogenic variants that can aid in their interpretation of newly observed VUS. CRISPR-assisted gene targeting is also expected to accelerated the pace at which mouse models can be generated [67 68].

The path to clinical implementation will necessarily involve the inclusion of clinically calibrated functional data into comprehensive multifactorial statistical models. Functional assays are specialized tests that most diagnostic testing laboratories are currently not able to provide. In the US, results from such tests cannot be directly used for clinical decisions unless they are conducted under Clinical Laboratory Improvement Amendments (CLIA) guidelines. However, if data have been validated (i.e. evaluated for a comprehensive series of performance metrics such as sensitivity, specificity, accuracy, precision, repeatability, and robustness), functional assays reported from research labs can be used as evidence for clinical annotation [7]. It is important to consider that mistakes may occur in the processing of a sample in the absence of standard operating procedures. Those mistakes include clerical mistakes in reports (e.g. reporting pathogenic when it should be non-pathogenic), sample swapping, errors in measurements due to lack of equipment calibration or improper staff training. Thus, evidence from functional studies performed in a research setting must be carefully verified to determine data quality, reliability and the degree of confidence in the results. In the context of risk assessment, we caution against the use of functional data as the sole source of information for clinical recommendations. Despite these challenges, the value of functional assessment of variants to improve cancer care, based on international and multidisciplinary collaborations, is expected to be high and therefore we envisage the clinical implementation of functional assays for classification of VUS to proceed at an accelerated pace.

Abbreviations

BIC

Breast Cancer Information Core

BRCT

BRCA1 C-terminal domain

HBOC

hereditary breast and ovarian cancer

HR

homologous recombination

IARC

International Agency for Research on Cancer

OR

odds ratio

PARP

poly ADP ribose polymerase

RR

relative risk

VUS

variant of uncertain clinical significance