Familial Risk Stratification Across Cancer Syndromes Using Fam3PRO

Abstract

Purpose:

Quantitative assessment of the risk of inherited cancer susceptibility should reflect the growing number of well-documented gene-cancer associations beyond single syndromes, to increase efficiency when identifying candidates for genetic testing and early detection.

Methods:

We validate Fam3PRO, a computationally efficient Mendelian risk prediction methodology and platform that supports constructing models with an arbitrary number of genes and cancers, on three independent multi-ethnic panel cohorts. Fam3PRO was trained using population-level parameters from existing literature for 21 genes and 17 cancers.

Results:

Fam3PRO provides discrimination and calibration comparable to the widely-adopted syndrome-specific models BRCAPRO and MMRpro, for genes associated with Breast-Ovarian and Lynch syndromes. Furthermore, when assessing the probability of being heterozygous for at least one pathogenic variant in any of the 21 genes, Fam3PRO has a discrimination of 0.64 (95% C.I. 0.62–0.67) and a calibration (observed divided by expected) of 1.13 (95% C.I. 1.05–1.22) in the combined cohort. At probability thresholds of 2.5% and 5%, Fam3PRO identifies more individuals at high risk of being heterozygous for pathogenic variants in any of the 21 genes than BRCAPRO and MMRpro.

Conclusion:

Fam3PRO provides a validated approach for familial risk stratification across a broad spectrum of cancer types, including estimates of carrier probabilities and future cancer risk.

Introduction

A cornerstone of early detection is identifying individuals at sufficient risk to benefit from genetic screening¹. Familial risk remains a central component of risk stratification^2–5. Its potential is further amplified by the drop in germline testing cost and the increase in the number and type of evaluable pathogenic germline variants (PGVs).

Strong evidence has emerged that a diversity of inherited genetic changes explain syndromes once thought to be distinct, as the pleiotropy of hereditary factors that increase risk of multiple cancers is better recognized⁶. Increasing use of multi-gene panel tests probing more than 100 genes simultaneously has revealed a broad spectrum of variants beyond the expectations of expert clinical opinion and the limited set of genes initially incorporated into syndrome-specific risk models. A growing body of literature suggests that many PGVs increase the risk of multiple cancers^7–12.

In parallel, the potential clinical utility of familial risk stratification is amplified in the rapidly evolving landscape of Multi-Cancer Detection (MCD) tests^1,13–15. While they are no replacement for syndrome-specific early detection modalities, MCD tests are a powerful technology that can detect the presence of cancer from multiple potential origin sites in asymptomatic individuals. Although their specificity can be notable, positive predictive values (PPVs) are modest if applied in unselected populations and improve considerably when applied to moderate or high-risk strata¹⁶. Eligibility for both MCD and germline panel tests strongly emphasizes the need for familial risk stratification that considers multiple types of cancer. Additionally, pre-test probabilities are used to determine insurance coverage, and future cancer risk estimates can aid clinical management and decision-making for those with family history who receive negative or variant of uncertain significance (VUS) panel testing results.

Quantitative risk measures are widely used to integrate clinical guidelines in prevention, early detection, and genetic counseling^17,18. According to the National Comprehensive Cancer Network (NCCN) criteria for germline testing, patients with pre-test probabilities greater than 5% of a BRCA1 (HGNC:1100) or BRCA2 (HGNC:1101) PGV based on models such as BRCAPRO^19,20, IBIS²¹ (Tyrer-Cuzick), or BOADICEA²² qualify for genetic testing; those with a 2.5% probability can also be considered for testing²³. Similarly, individuals for whom MMRpro²⁴, PREMM5²⁵, MMRPredict²⁶ and PREMM126²⁷ estimate a sufficiently high probability of being heterozygous for a PGV of an MMR gene (above 2.5 or 5%) are recommended for Lynch syndrome panel testing²⁸. These models, as well as others not explicitly incorporated in NCCN guidelines, are generally limited to specific gene families and syndromes and do not account for recently identified associations between PGVs in dozens of genes across various cancers.

These challenges motivate risk stratification models that can flexibly incorporate more gene-cancer associations and provide a scalable approach for future expansions. Thus, we developed Fam3PRO (formerly PanelPRO²⁹), a computationally efficient platform for Mendelian risk that can incorporate an arbitrary number of genes and cancers and addresses both carrier probabilities for any desired subset of susceptibility genes and future risk for all relevant cancers. Here and throughout the manuscript, we will use the word “carrier” to refer to an individual who is heterozygous for a PGV, e.g. “carrier probability” is the probability of being heterozygous for a PGV on a given gene. We will use “non-carrier” to refer to an individual who is not heterozygous for any PGV; when evaluating Fam3PRO, the “non-carrier” definition is further restricted to individuals who are not heterozygous for a PGV on any of the 21 genes in package v1.1.0 and excludes those who are heterozygous for one or more PGVs (but not on any Fam3PRO gene) or that have a VUS on any Fam3PRO gene. Previously, we presented our implementation of the R software package, describing the user workflow using synthetic examples and providing computational benchmarks²⁹, and gave a statistically rigorous treatment of the methodology³⁰. The focus of this article is the validation of Fam3PRO’s carrier probabilities on three carefully curated independent cohorts from the United States and Latin America, comprising a total of 20,781 families.

Materials and Methods

Via a comprehensive literature review and analysis, we built Fam3PRO²⁹, summarized in Supplemental Section 1, and trained the required input parameters to cover 21 genes (ATM (HGNC:795), BARD1 (HGNC:952), BRCA1 (HGNC:1100), BRCA2 (HGNC:1101), BRIP1 (HGNC:20473), CDH1 (HGNC:1748), CDK4 (HGNC:1773), CDKN2A[P16] (HGNC:1787), CHEK2*1100delC (HGNC:16627), EPCAM (HGNC:11529), MLH1 (HGNC:7127), MSH2 (HGNC:7325), MSH6 (HGNC:7329), NBN*657del5 (HGNC:7652), PALB2 (HGNC:26144), PMS2 (HGNC:9122), PTEN (HGNC:9588), RAD51C (HGNC:9820), RAD51D (HGNC:9823), STK11 (HGNC:11389), and TP53 (HGNC:11998)) and 17 cancers (brain, breast, colorectal, endometrial, gastric, hepatobiliary, kidney, leukemia, melanoma, ovarian, osteosarcoma, pancreas, prostate, small intestine, soft tissue sarcoma, thyroid, and urinary bladder), expanding on an earlier 11-gene, 11-cancer input specification³⁰ (Figure 1). The full Human Genome Variation Society (HGVS) coding descriptions for the CHEK2*1100delC and NBN*657del5 variants are NM 007194.4:c.1100del and NM 002485.5:c.657 661del, respectively. CDKN2A[P16] refers specifically to CDKN2A PGVs that affect encoding of the p16(INK4A) protein.

Figure 1: A map of the pathogenic effects considered in Fam3PRO, including the 17 cancers (rows) and 21 genes (columns).

Orange cells correspond to effects previously included in the BayesMendel package⁵² via BRCAPRO, MMRPRO and MelaPRO. Blue cells correspond to effects unique to Fam3PRO. See the Fam3PRO software documentation or Supplemental Section 2 for further details on the external estimates used to train the model, including additional information on the specific risks for each gene-cancer association.

Mendelian models like Fam3PRO use principles of Mendelian genetics, Bayesian probability theory, and gene-specific knowledge^31,32. They are trained using external estimates of PGV allele frequencies, age-dependent probabilities of developing cancer conditional on genotype (cancer penetrances), and other information from existing literature. As user input, they accept a detailed pedigree of an individual’s family history (including genealogical relationships, age, sex, cancer diagnosis status and age at diagnosis, and additional risk-modifying information such as tumor biomarker testing results), and they return both an individual’s set of gene-specific carrier probabilities and future risk of developing various cancers. See the software documentation or Supplemental Section 2 for further details.

We validated Fam3PRO (package v1.1.0) in three independent cohorts not used for training: 1) the USC-Stanford Hereditary Cancer Panel (HCP) Testing Study³³ comprising 2000 racially and socioeconomically diverse participants from three medical centers; 2) the Center for Cancer Risk Assessment at Massachusetts General Hospital (MGH) cohort; and 3) the City of Hope (COH) cohort collected by the Clinical Cancer Genomics Community Research Network (CCGCRN), a group of over 40 collaborating sites in the United States and Latin America. All three cohorts select individuals with family history of cancer. HCP counselees were tested using fixed 25- or 28-gene panels that covered all 21 genes considered in our validation. We previously used data from the HCP cohort to validate an 11-gene, 11-cancer model³⁰. MGH and COH counselees were tested using a broad range of panels; some panels included a subset of the 21 genes, while others included additional genes.

We cleaned each cohort’s data and imputed missing information when possible, as described in Supplemental Sections 3.1–3.3. We subsequently applied to all cohorts our shared exclusion criteria:

1. The counselee is heterozygous for one or more PGVs, but not on any Fam3PRO gene.

2. The counselee has a VUS on any Fam3PRO gene.

3. The counselee was not tested for any of the Fam3PRO genes.

4. The pedigree input file is not compatible with Fam3PRO due to pedigree loops or other model errors.

Regarding the second criterion: while appropriate handling of VUSs is an ongoing challenge, we elected to exclude counselees with VUSs on Fam3PRO genes because retaining and treating them as non-carriers (not heterozygous for a PGV on any Fam3PRO gene) seemed too strong of an assumption: VUSs may later be reclassified as pathogenic. After exclusions, we have 1,274 remaining counselees in HCP; 2,239 in MGH, 555 of which reported testing for all 21 genes; and 17,268 in COH, 3,493 of which reported testing for all 21 genes (Supplemental Table 1). Table 1 summarizes demographic data for the three cohorts, for the 5,322 counselees who were tested for all 21 genes. Table 2 summarizes their PGV carrier statuses, Supplemental Table 2 summarizes cancer diagnoses for affected counselees and relatives, and Supplemental Table 3 summarizes available tumor biomarker testing results for counselees. In brief, while a broad spectrum of genes and syndromes are represented in the validation data, breast was by far the most common cancer site (45.5% of counselees and 5.5% of all individuals in the combined cohort), followed by colorectal (11.5% of counselees and 1.9% of all individuals) (Supplemental Table 2). The highest PGV frequencies were 1.3% for ATM, 2% for BRCA1, 2% for BRCA2, 1.8% for CHEK2*1100delC, and 1.3% for MLH1 in the combined cohort (Table 2). PGV frequencies were <1% for all other genes. These real-world cohorts reflect common testing indications in clinical settings, such as being strongly skewed toward female counselees: only 13.8% of counselees were male in the combined cohort, and the MGH cohort had only 0.9% males (Table 1).

Table 1: Demographic characteristics of the HCP, MGH, COH, and combined cohorts, restricted to counselees who were tested for all 21 genes.

The COH cohort recruited counselees from sites in both the United States and Latin America; all counselees in the HCP and MGH cohorts were recruited in the United States.

	HCP	MGH	COH	Combined
Number of counselees	1274	555	3493	5322
Mean family size	34.14	27.57	35.06	34.06
% male counselees	18.9%	0.9%	14.1%	13.8%
Mean counselee age (years)	51.48	56.99	51.51	52.07
% Ashkenazi Jewish counselees	6.6%	23.6%	6.3%	8.2%
% Italian counselees	0%	18.2%	0%	1.9%
% American Indian and Alaska Native counselees	0.3%	0.2%	0.4%	0.3%
% Asian counselees	8.6%	3.2%	6.9%	6.9%
% Black counselees	3.3%	1.1%	1.7%	2%
% Hispanic counselees	39.8%	3.1%	17.3%	21.2%
% White Hispanic counselees	0%	3.4%	8.5%	5.9%
% White Non-Hispanic counselees	43.2%	76.4%	43.1%	46.6%
% White counselees	0.2%	0%	3.4%	2.3%
% Latin American counselees	0%	0%	12.2%	8%

Table 2: Number (percent) of counselees who are known heterozygotes of Fam3PRO PGVs in the HCP, MGH, COH, and combined cohorts, restricted to counselees who were tested for all 21 genes.

It is possible for a counselee to be heterozygous for more than one PGV; counselees may also be heterozygotes of PGVs for genes not shown in this table. Genes shown are: ATM (HGNC:795), BARD1 (HGNC:952), BRCA1 (HGNC:1100), BRCA2 (HGNC:1101), BRIP1 (HGNC:20473), CDH1 (HGNC:1748), CDK4 (HGNC:1773), CDKN2A[P16] (HGNC:1787), CHEK2*1100delC (HGNC:16627), EPCAM (HGNC:11529), MLH1 (HGNC:7127), MSH2 (HGNC:7325), MSH6 (HGNC:7329), NBN*657del5 (HGNC:7652), PALB2 (HGNC:26144), PMS2 (HGNC:9122), PTEN (HGNC:9588), RAD51C (HGNC:9820), RAD51D (HGNC:9823), STK11 (HGNC:11389), and TP53 (HGNC:11998). “Any Gene” and “Non-carrier” refer to carrier status with respect to the 21 genes in Fam3PRO.

	HCP	MGH	COH	Combined
ATM	10 (0.8%)	9 (1.6%)	52 (1.5%)	71 (1.3%)
BARD1	1 (0.1%)	3 (0.5%)	2 (0.1%)	6 (0.1%)
BRCA1	32 (2.5%)	11 (2%)	66 (1.9%)	109 (2%)
BRCA2	27 (2.1%)	9 (1.6%)	73 (2.1%)	109 (2%)
BRIP1	2 (0.2%)	7 (1.3%)	8 (0.2%)	17 (0.3%)
CDH1	1 (0.1%)	1 (0.2%)	9 (0.3%)	11 (0.2%)
CDK4	0 (0%)	0 (0%)	0 (0%)	0 (0%)
CDKN2A[P16]	1 (0.1%)	1 (0.2%)	4 (0.1%)	6 (0.1%)
CHEK2*1100delC	13 (1%)	15 (2.7%)	67 (1.9%)	95 (1.8%)
EPCAM	0 (0%)	0 (0%)	0 (0%)	0 (0%)
MLH1	7 (0.5%)	3 (0.5%)	58 (1.7%)	68 (1.3%)
MSH2	8 (0.6%)	1 (0.2%)	27 (0.8%)	36 (0.7%)
MSH6	7 (0.5%)	3 (0.5%)	14 (0.4%)	24 (0.5%)
NBN*657del5	2 (0.2%)	5 (0.9%)	7 (0.2%)	14 (0.3%)
PALB2	7 (0.5%)	4 (0.7%)	29 (0.8%)	40 (0.8%)
PMS2	6 (0.5%)	6 (1.1%)	16 (0.5%)	28 (0.5%)
PTEN	0 (0%)	0 (0%)	1 (0%)	1 (0%)
RAD51C	1 (0.1%)	1 (0.2%)	9 (0.3%)	11 (0.2%)
RAD51D	2 (0.2%)	1 (0.2%)	8 (0.2%)	11 (0.2%)
STK11	0 (0%)	0 (0%)	2 (0.1%)	2 (0%)
TP53	2 (0.2%)	2 (0.4%)	32 (0.9%)	36 (0.7%)
Any Gene	128 (10%)	76 (13.7%)	471 (13.5%)	675 (12.7%)
Non-carrier	1146 (90%)	479 (86.3%)	3022 (86.5%)	4647 (87.3%)

We evaluated Fam3PRO’s carrier probabilities and compared them to previous syndrome-specific BayesMendel models BRCAPRO, MMRpro, and MelaPRO. For each model, we evaluated³⁴: root mean square error (RMSE) between the predicted and observed results; area under the receiver operating characteristic curve (AUC) as a measure of discrimination; observed divided by expected (O/E) number of heterozygotes of a PGV as a measure of calibration (bias is indicated by a deviation from 1.0); and sensitivity, specificity, and PPV at 2.5% and 5% thresholds. When computing these metrics, non-carriers were defined as individuals without a PGV in any Fam3PRO gene, and metrics were calculated over non-carriers and those who are only heterozygous for a PGV in the gene/gene set of interest. We only report results for genes with at least 10 counselees who are heterozygous for a PGV. We obtained confidence intervals (C.I.s) from 1000 bootstrap samples of counselees. We also examined performance differences between Fam3PRO and syndrome-specific submodels via the fraction of bootstrap replicates where metrics for Fam3PRO improve over the submodel.

In the main text, we report results based on the 5,322 counselees who were tested for all 21 genes. In Supplemental Section 5 (Supplemental Tables 5–8, Supplemental Figures 3–5), we report results on an expanded cohort of counselees tested for at least one, but not necessarily all 21 genes. We evaluated gene-specific metrics based on counselees with testing results for the gene considered. As an additional sensitivity check, we also report results that retain the 1,915 counselees that were tested for all 21 genes and are heterozygous for one or more PGVs, but not on any Fam3PRO gene (the first exclusion criterion) or that have a VUS on a Fam3PRO gene (the second exclusion criterion) (Supplemental Figures 6–8). When evaluating performance metrics, these additional counselees were grouped with the non-carriers.

Results

Performance of Probabilistic Prediction

We applied Fam3PRO to the three cohorts, separately and together. Figure 2 and Supplemental Figures 1–2 summarize the results in counselees tested for all 21 genes. Panels correspond to metrics, rows to tasks (genes and gene sets being evaluated as outcomes), and colors to cohorts. Generally, metrics do not vary systematically by cohort, suggesting generalizability of prediction performance across populations³⁵.

Figure 2: RMSE, AUC, and calibration for Fam3PRO carrier probabilities evaluated on the HCP, MGH, COH, and combined cohorts, restricted to counselees who were tested for all 21 genes.

The “Any” (any of the 21 genes in Fam3PRO), “BayesMendel” (BRCA1, BRCA2, CDKN2A[P16], MLH1, MSH2, MSH6), “BRCAPRO” (BRCA1, BRCA2), and “MMRpro” (MLH1, MSH2, MSH6) rows refer to model performance for predicting heterozygotes of PGVs for any gene in the gene set. CDKN2A refers to PGVs affecting P16 protein transcription, CHEK2 refers to the 1100delC PGV, and NBN refers to the 657del5 PGV. Metrics are only reported when there are at least 10 counselees who are heterozygous for a PGV of the given gene or gene set. 95% bootstrap confidence intervals are also shown.

The gene-specific RMSEs for the combined cohorts are never above 0.2, indicating that on average, the predictions are less than 0.2 points away from the binary true carrier status on the probability scale. As expected, RMSE increases as genes are combined. The MMRpro gene set, which includes Lynch syndrome genes MLH1, MSH2, and MSH6, has RMSE 0.15 and the BRCAPRO set (BRCA1, BRCA2) has RMSE 0.22. As the cancer spectra of the gene set grows, so does the RMSE: it is 0.26 for the BayesMendel set (BRCA1, BRCA2, CDKN2A[P16], MLH1, MSH2, MSH6). Predicting the presence of a PGV in any of the 21 genes is the hardest task and has the worst RMSE (0.34).

Gene-specific discrimination, as measured by AUC, varies across genes, with broad C.I.s in many cases. Except for PMS2, AUCs for all individual genes and cohorts are above 0.5, with excellent discrimination for the other Lynch syndrome genes. The AUC is generally higher for more penetrant genes associated with fewer cancers compared to genes associated with more cancers or moderate penetrances. In the combined and separate cohorts, the AUCs for BRCA1 are 0.78–0.84, while those for BRCA2 are 0.68–0.72. Both genes are highly penetrant for similar cancers. However, in our data, the average number of relatives with ovarian cancer is higher among heterozygotes of BRCA1 PGVs (0.89) compared to heterozygotes of BRCA2 PGVs (0.36), the latter being similar to that of non-carriers (0.27). Discrimination for the presence of a PGV in any of the 21 genes is 0.64 overall and over 0.6 for two of the three cohorts.

No systematic biases are apparent for gene-specific tasks in the combined cohort: the C.I.s of the O/E ratios all cover the target value of 1.0 except for ATM and CHEK2*1100delC, which show statistically significant biases toward underprediction, and CDH1 and MSH2, which show statistically significant biases toward overprediction. 11 of the gene-specific point estimates for the combined cohort are above 1.0, while 4 are below; calibration is approximately on target for predicting the presence of a PGV in any of the 21 genes (1.13) and for predicting any of the genes in the BayesMendel (0.98), BRCAPRO (0.97), and MMRpro (0.99) gene sets. The calibration for the “Any” task is very good in the HCP cohort (0.96); a minor bias is seen in COH (1.14) and a more pronounced bias is seen in MGH (1.54), although the C.I.s for the O/E ratio overlap between cohorts. As an alternative presentation, Supplemental Table 4 reports the observed percent and mean predicted carrier probability (“expected” percent) of counselees that are heterozygous for each PGV. Observed and expected percents for “Any” gene are very close for HCP (10% vs. 10.4%), show evidence for underprediction in MGH (13.7% vs. 8.9%) and are intermediate for COH and the combined cohort (13.5% vs. 11.8% and 12.7% vs. 11.2%, respectively).

In the expanded cohort, the RMSEs and O/Es are generally higher, while discrimination remains similar (Supplemental Figure 3). Results for individual genes and the BRCAPRO set are nearly identical with and without retaining counselees with only PGVs on non-Fam3PRO genes or VUSs on Fam3PRO genes (Supplemental Figures 6–8). However, the cohort without these additional exclusions has greater RMSEs, loses discrimination and sensitivity, and underpredicts relative to the main analysis, for the “Any”, BayesMendel, and MMRpro genes sets. These sensitivity analyses underscore the difficulties of working with real-world data, where counselees are tested using a variety of panels and with many reported VUSs. Establishing reasonable criteria for fair comparisons between “carriers” and “non-carriers” is therefore challenging and comes with trade-offs.

The functionality of Fam3PRO subsumes that of the BayesMendel models BRCAPRO, MMRpro, and MelaPRO. For most applicable genes (BRCA1, BRCA2, CDKN2A[P16], MLH1, MSH2, MSH6), the results remain highly consistent between tools, as intended. In the combined cohort, the RMSEs and AUCs for individual genes and gene sets typically differ by less than 0.01. Sensitivity, specificity, and PPV typically differ by less than 0.04. While most tasks also had similar calibration, Fam3PRO overpredicts for MLH1 (O/E of 0.95) while MMRpro underpredicts (O/E of 1.05); there are also large differences for MSH2 (0.67 for Fam3PRO and 0.74 for MMRpro) and MSH6 (1.19 for Fam3PRO and 2.35 for MMRpro). Calibration for the MMRpro gene set reflects this disparity (0.99 for Fam3PRO and 1.16 for MMRpro). Supplemental Figures 9–11, 13–15, 17–19, and 21–23 and Supplemental Tables 9–11, 13–15, 17–19, and 21–23 compare metrics for Fam3PRO and the BayesMendel submodels, applied to the separate and combined cohorts, and restricted to the set of counselees for whom all 21 genes were tested.

We also evaluated performance differences between Fam3PRO and the BayesMendel submodels via the proportion of the 1000 bootstrap replicates where the performance metrics for Fam3PRO improve over the relevant submodel (Supplemental Tables 12, 16, 20, and 24). Overall, improvement trends were not consistent in either direction, across genes/gene sets, performance metrics, or cohorts. For example, Fam3PRO has better sensitivity at 5% for 100% of the bootstraps when evaluating individual genes and the BRCAPRO set in the MGH cohort, but is mixed for the MMRpro and BayesMendel gene sets in MGH, as well as across genes/gene sets in HCP and COH.

Sensitivity at Clinically-Relevant Cutoffs

Figure 3 visualizes the value added of multi-syndrome prediction compared to single-syndrome in terms of sensitivity. It considers heterozygotes of any PGV in the combined cohort (restricted to those tested for the 21 genes) and plots the probabilities of being heterozygous for a PGV of any Fam3PRO gene against the probabilities of being heterozygous for a PGV of any BRCAPRO gene and any MMRpro gene. Among the 675 counselees in the combined cohort who are heterozygous for any PGV, 360 and 253 have carrier probabilities above 2.5% and 5% for Fam3PRO but not BRCAPRO, respectively; none have carrier probabilities above 2.5% or 5% for BRCAPRO but not Fam3PRO. Overall, sensitivity is 0.85 at 2.5% (0.61 at 5%) for any Fam3PRO gene and 0.58 at 2.5% (0.47 at 5%) for any BRCAPRO gene. Similarly, 463 and 317 counselees have carrier probabilities above 2.5% and 5%, respectively, for Fam3PRO but not MMRpro; in the opposite direction, 0 have carrier probabilities above 2.5% and 4 have carrier probabilities above 5%. Sensitivity is 0.7 at 2.5% (0.63 at 5%) for any MMRpro gene. Supplemental Figures 12, 16, and 20 reproduce Figure 3 separately for the HCP, MGH, and COH cohorts. Fam3PRO also detects far more true heterozygotes for PGVs than syndrome-specific models for each separate cohort.

Figure 3: Among combined cohort counselees with at least one PGV, carrier probabilities for PGVs of any Fam3PRO gene are plotted against carrier probabilities for PGVs of any BRCAPRO gene (BRCA1 or BRCA2, left) and against carrier probabilities for PGVs of any MMRpro gene (MLH1, MSH2, or MSH6, right).

All counselees shown as points were tested for all 21 genes, colored blue for heterozygotes of BRCAPRO PGVs, orange for heterozygotes of MMRpro PGVs, and black for heterozygotes of all other Fam3PRO PGVs. Reference lines are drawn at 5%.

Of the 675 combined cohort counselees who are true heterozygotes of a PGV in any Fam3PRO gene, 360 and 253 have carrier probabilities above a 2.5% and 5% threshold for Fam3PRO but not BRCAPRO, respectively; 0 have carrier probabilities above 2.5% or 5% for BRCAPRO but not Fam3PRO. 463 and 317 counselees have carrier probabilities above 2.5% and 5%, respectively, for Fam3PRO but not MMRpro; in the opposite direction, 0 have a carrier probability above 2.5% and 4 have carrier probabilities above 5%.

Of the 215 true heterozygotes for BRCAPRO PGVs (blue), 73 and 59 have carrier probabilities above a 2.5% and 5% threshold for Fam3PRO but not BRCAPRO, respectively; 0 have carrier probabilities above 2.5% or 5% for BRCAPRO but not Fam3PRO. Of the 127 true heterozygotes for MMRpro PGVs (orange), 25 and 20 counselees have carrier probabilities above 2.5% and 5%, respectively, for Fam3PRO but not MMRpro; 0 and 3 have carrier probabilities above 2.5% and 5% for MMRpro but not Fam3PRO.

The roughly linear pattern along the main diagonal of each of these figures suggests that the additional heterozygotes for PGVs flagged are not due to an overall inflation of the predicted probabilities compared to earlier models. Rather, most of the additional heterozygotes flagged are in the narrow top-left quadrant corresponding to low BRCAPRO/MMRpro but moderate or high Fam3PRO probabilities. The wide range of Fam3PRO probabilities in this quadrant reflects the importance of considering syndrome overlap when identifying candidates for testing. In addition, predictions cluster more closely along the main diagonal in the BRCAPRO panel compared to MMRpro. Compared to BRCAPRO, Fam3PRO considers two additional cancers (pancreas and prostate) for BRCA1 and BRCA2. In contrast, it considers six additional cancers compared to MMRpro (Figure 1), resulting in the greater dispersion of points above the diagonal.

Increased sensitivity may come at the expense of increased false positives. However, we observe for the combined cohort that PPV is 0.11 at 2.5% (0.14 at 5%) for the BRCAPRO gene set and 0.19 at 2.5% (0.22 at 5%) for the MMRpro gene set; Fam3PRO’s PPV for “Any” gene is intermediate at 0.15 at 2.5% (0.17 at 5%).

Discussion

Identifying individuals at increased cancer risk from heritable PGVs is important for clinical and research efforts in early detection and prevention. Mendelian models offer a systematic, principled way to integrate knowledge of genetic susceptibility and estimate carrier probabilities and future cancer risk for use in decision support. The Fam3PRO version considered here is the most advanced of these models, covering 21 genes and 17 cancers. In this article, we validated Fam3PRO carrier probabilities in three independent clinical cohorts from the United States and Latin America, comprising 20,781 families. We evaluated single-gene tasks, gene sets associated with existing syndrome-specific models, and prediction of being heterozygous for a PGV in at least one of 21 genes (the “Any” task).

Calibration is consistently good for syndrome-specific sets, though we notably observe biases for CHEK2*1100delC and in the MGH cohort for the “Any” task. As expected, discrimination varies across tasks. Better discrimination is likely associated with higher penetrance, association with less common cancers, association with cancers affected by fewer other genes, age of onset differences compared to sporadic cases, and overall number of cancers affected. Discrimination for the syndrome-specific gene sets is strong, supporting the clinical use of Fam3PRO for these subtasks.

As expected, variety in the strength and type of susceptibility conferred by the 21 genes makes it challenging to discriminate non-carriers from heterozygotes of “Any” PGV. All three cohorts are, to varying extents, ascertained based on criteria that indicate familial susceptibility, so discrimination is in addition to what these criteria confer. Fam3PRO’s ability to predict carriage over and above ascertainment criteria supports its utility for identifying additional at-risk individuals (Figure 3). Compared to syndrome-specific approaches used for NCCN guidelines, Fam3PRO flags many more true heterozygotes of PGVs as high-risk without compromising PPV.

Fam3PRO’s comprehensive approach can be used to better identify individuals who should undergo panel genetic testing, MCD testing, and other early detection modalities. Fam3PRO substantially expands beyond existing syndrome-specific approaches that focus on a limited number of genes and cancers, such as the BayesMendel models. Simultaneously, we designed Fam3PRO to perform similarly to existing BayesMendel models when presented with the same task and input, to facilitate adoption by current users. These existing models are validated^24,36–38 and widely used in at least six commercial clinical tools. For the cohorts presented, we verified that Fam3PRO retains overall calibration and discrimination for the same syndrome-specific gene sets considered by the legacy BayesMendel models. Furthermore, Fam3PRO’s performance in gene-specific tasks is consistent with existing Mendelian models for the six genes included in both, indicating that the increased flexibility and multi-syndrome functionality of Fam3PRO did not come to the detriment of performance in syndrome-specific tasks.

Other Mendelian risk prediction models for breast and ovarian cancer susceptibility include Claus³⁹, IBIS (Tyrer-Cuzick)²¹, and BOADICEA^22,40,41, which also consider polygenic risk scores and other breast cancer risk factors. All focus on a single syndrome or cancer type. Non-Mendelian models for carrier probabilities and future risk of cancer are also generally syndrome-specific^{24–26,42–46}. An important step towards developing non-Mendelian multi-syndrome risk stratification methods is PREMMplus⁴⁷. The PREMMplus Q score combines three submodels: one trained to predict the carrier status of 7 genes (APC, MUTYH (biallelic), and Lynch syndrome genes EPCAM, MLH1, MSH2, MSH6, and PMS2), one trained to predict BRCA1 and BRCA2, and one trained to predict TP53. The submodels are evaluated separately on maternal and paternal family history summaries that include up to 18 cancer types, for six model evaluations in total. They are combined by selecting the maximum score over family sides per submodel and then applying a multiplicative rule to combine the selected scores. These details highlight the differences between PREMMplus and models built on the Fam3PRO platform²⁹: 1) Fam3PRO provides pre-test carrier probabilities as well as post-test cancer risk, while PREMMplus only provides the former; 2) Fam3PRO, as all Mendelian models, provides probabilities, while PREMMplus’ Q is a non-probabilistic score; 3) Fam3PRO considers the overlap of traditional syndromes, while PREMMplus’ Q, because of the multiplicative rule described above, does not acknowledge this overlap.

Fam3PRO makes several modeling assumptions³⁰ and is limited by uncertainties in its inputs. Allele frequencies and penetrances in existing literature may not be robust for rare genes and cancers, and user-specified pedigrees may contain errors or missing information⁴⁸. As the number of cancers included in Fam3PRO grows, collecting complete, high-quality family history information can be challenging^49,50. While in practice, probability thresholds should be motivated by clinical context and constraints, we evaluated sensitivity, specificity, and PPV at 2.5% and 5% cutoffs in part to facilitate interpretation for readers familiar with the current NCCN recommendations for breast-ovarian and Lynch syndrome models^23,28.

Our validation cohorts ascertain individuals at familial risk. While ascertainment criteria likely mimic those for clinical populations where Fam3PRO would be applied, we observe a robust representation of women and PGVs associated with hereditary breast and ovarian cancer syndrome, particularly in the MGH cohort. As such, this needs to be considered when weighing the generalizability of results, especially in the context of MCD approaches and as we move beyond the existing syndrome-specific paradigm. We are not yet able to quantify performance in unselected individuals. Previous evidence⁵¹ and common sense suggest that discrimination could improve considerably in unselected populations, because of the higher number of moderate-risk individuals, who are easier to discriminate from those with familial risk. However, calibration may suffer, as it relies primarily on estimated prevalence of susceptibility genes, some of which are relatively rare and inadequately characterized to date.

In summary, Fam3PRO provides a robust response to the pressing need for risk stratification in support of germline panel testing and MCD testing. It is validated in large multi-ethnic cohorts and provides sufficient discrimination for important clinical applications.

Data Availability

The Data User Agreement and IRB do not allow for data sharing. Software for the Fam3PRO model, including the model parameter database, is available at: https://projects.iq.harvard.edu/bayesmendel/panelpro. Code to reproduce the findings reported in this paper can be found at: https://github.com/bayesmendel/Fam3PROValidation.