Prediction of contralateral breast cancer: external validation of risk calculators in 20 international cohorts

Abstract

Background

Three tools are currently available to predict the risk of contralateral breast cancer (CBC). We aimed to compare the performance of the Manchester formula, CBCrisk, and PredictCBC in patients with invasive breast cancer (BC).

Methods

We analyzed data of 132,756 patients (4682 CBC) from 20 international studies with a median follow-up of 8.8 years. Prediction performance included discrimination, quantified as a time-dependent Area-Under-the-Curve (AUC) at 5 and 10 years after diagnosis of primary BC, and calibration, quantified as the expected-observed (E/O) ratio at 5 and 10 years and the calibration slope.

Results

The AUC at 10 years was: 0.58 (95% confidence intervals [CI] 0.57–0.59) for CBCrisk; 0.60 (95% CI 0.59–0.61) for the Manchester formula; 0.63 (95% CI 0.59–0.66) and 0.59 (95% CI 0.56–0.62) for PredictCBC-1A (for settings where BRCA1/2 mutation status is available) and PredictCBC-1B (for the general population), respectively. The E/O at 10 years: 0.82 (95% CI 0.51–1.32) for CBCrisk; 1.53 (95% CI 0.63–3.73) for the Manchester formula; 1.28 (95% CI 0.63–2.58) for PredictCBC-1A and 1.35 (95% CI 0.65–2.77) for PredictCBC-1B. The calibration slope was 1.26 (95% CI 1.01–1.50) for CBCrisk; 0.90 (95% CI 0.79–1.02) for PredictCBC-1A; 0.81 (95% CI 0.63–0.99) for PredictCBC-1B, and 0.39 (95% CI 0.34–0.43) for the Manchester formula.

Conclusions

Current CBC risk prediction tools provide only moderate discrimination and the Manchester formula was poorly calibrated. Better predictors and re-calibration are needed to improve CBC prediction and to identify low- and high-CBC risk patients for clinical decision-making.

Introduction

A rising number of women with breast cancer (BC) are at risk to develop a new primary tumor in the contralateral breast (CBC) with consequently another cancer treatment and potentially less favorable prognosis [1]. Although CBC incidence is low (~ 0.4% per year) in the general BC population, contralateral preventive mastectomy (CPM) is increasing, also among women with low-CBC risk [2–5].

Three tools are tools currently available to predict the risk of CBC, although probably none are widely used: (1) the Manchester formula; (2) CBCrisk, and (3) PredictCBC [6–8]. The Manchester group in the United Kingdom (UK) proposed a set of guidelines for counseling women about CPM [8]. Based on a systematic review of the literature, they devised a formula to estimate lifetime CBC risk based on age at first primary BC, family history of BC, estrogen-receptor (ER) status, diagnosis of ductal carcinoma in situ (DCIS), and oophorectomy.

The second tool, CBCrisk, was developed using data on 1921 CBC cases and 5763 matched controls with primary BC [7]. The model uses data on age at first BC diagnosis, age at first birth, first degree family history of BC, high-risk pre-neoplasia, breast density (obtained using the BI-RADS system), ER status, first BC type (pure invasive, pure DCIS, a mix of the two, unknown), and adjuvant endocrine therapy. External validation was performed using two independent studies in the United States (US) of 5185 and 6035 patients with 111 and 117 CBC events [7, 9]. A web-based application provides individualized prediction of CBC risk [10].

Third, PredictCBC was developed, cross-validated and evaluated using data from 132,756 patients with first BC and 4672 CBC events, as part of an international collaboration [5]. PredictCBC predicts CBC risk as a function of family history (first degree) of primary BC, and information of primary BC diagnosis: age, nodal status, size, grade, morphology, ER status, human epidermal growth factor receptor 2 (HER2) status, administration of adjuvant or neoadjuvant chemotherapy, adjuvant endocrine therapy, adjuvant trastuzumab therapy, and radiotherapy. Two versions were developed: PredictCBC version 1A includes presence or absence of a mutation in the BRCA1 or BRCA2 genes, an important determinant of CBC [5, 11, 12], while PredictCBC version 1B was developed for untested patients.

External validation in different studies is relevant to assess the prediction performance of prediction models [13]. Our aim was to perform a head-to-head comparison between CBCrisk, PredictCBC and the Manchester formula. We hereto used several large population- and hospital-based studies used to develop and cross-validate the PredictCBC models.

Material and methods

External validation of CBCrisk and the Manchester formula was performed in 20 studies: four with individual patient data from the Netherlands [the Amsterdam Breast Cancer Study (ABCS), the Breast Cancer Outcome Study of Mutation carriers (BOSOM), the Erasmus MC Breast Cancer Registry (EMC), the Netherlands Cancer Registry (NCR)]; and 16 other studies of the Breast Cancer Association Consortium (BCAC). The latter is an international consortium of 102 studies comprising 182,898 patients (data version: January 2017) with a primary BC diagnosed between 1939 and 2016 [14]. Of these, 16 non-familial BC BCAC studies including invasive non-metastatic European-descent female patients with first primary invasive BC diagnosed from 1990 onwards, and with at least 10 CBC events, were included in the analyses [14]. Details about studies and patient selection, and data imputation were described previously [5].

The outcome was in situ or invasive metachronous CBC. Follow-up started 3 months after invasive first primary BC diagnosis, to exclude synchronous CBCs, and ended at date of CBC, distant metastasis (but not at loco-regional relapse), CPM or last date of follow-up (due to death, being lost to follow-up, or end of study), whichever occurred first. In the BCAC, 27,155 patients were recruited more than 3 months after diagnosis of the first primary BC (prevalent cases); for these patients, follow-up started at date of recruitment (left truncation). Distant metastasis and death due to any cause were competing events.

The Manchester formula provides an estimate of a woman’s individual lifetime CBC risk. To assess the prediction performance, we translated the lifetime CBC risk to 5- and 10-year CBC risks (see Supplementary Material). The predictors included in the CBC risk estimation in the Manchester formula, CBCrisk and PredictCBC models are provided in Table 1. Predictors that were sporadically missing were multiply imputed as described elsewhere [5].

Table 1.

Predictors included in current contralateral breast cancer risk prediction tools

List of predictors	CBCriskb	Manchester formulac	PredictCBC version 1Ad	PredictCBC version 1Bd
Age at diagnosis	✔	✔	✔	✔
Age at first birth	✔
First-degree family history	✔	✔	✔	✔
BRCA1/2 germline mutation		✔	✔
First breast cancer behavior typea	✔	✔
Lymph node status			✔	✔
Breast density	✔
Tumor size			✔	✔
Morphology			✔	✔
Tumor grade			✔	✔
High-risk pre-neoplasia	✔
ER status	✔	✔	✔	✔
HER2 status			✔	✔
Chemotherapy			✔	✔
Endocrine therapy	✔		✔	✔
Radiation to the breast			✔	✔
Trastuzumab			✔	✔
Oophorectomy under 40 years		✔

ER estrogen-receptor status, HER2 human epidermal growth factor receptor 2

^aContralateral breast cancer risk was calculated including women diagnosed with ductal carcinoma in situ

^bChowdhury et al. [7]

^cBasu et al. [8]

^dGiardiello et al. [5]

Statistical analysis

Discrimination, the ability of the model to differentiate between patients who experienced CBC and those who did not, was calculated by time-dependent Area-Underthe-Curve (AUCs) based on Inverse Censoring Probability Weighting at 5 and 10 years [15, 16]. Values of AUCs close to 1 indicate good discrimination while values close to 0.5 indicate poor discrimination (a coin flip). Calibration is the agreement between observed and predicted risk and is commonly characterized by calibration-in-the-large and slope statistic. Calibration-in-the-large characterizes the overall difference between the observed and predicted risks. It was calculated using the expected/observed (E/O) ratio. An E/O less than 1 indicates that the model systematically underestimates CBC risk, while an E/O above 1 indicates that the model systematically overestimates CBC risk. The expected number of cases was calculated by summing the individual predicted probabilities at 5 and 10 years, based on the patient-specific covariate values [17]. The observed number of cases was estimated by the non-parametric CBC cumulative incidence at 5 and 10 years. The calibration slope was estimated using a Fine and Gray regression model using the linear predictor of the prediction tools. The linear predictor was vs constructed as the sum of the factors included in each model weighted by the corresponding regression coefficients (or parameters), and then computed in the validation dataset exactly as reported for the development set. The calibration slope is determined as the regression coefficient for this linear predictor when fitted as a single covariate in a regression model of disease outcome in the validation dataset. A well-calibrated model should have a calibration slope of 1; slopes < 1 indicate that coefficients were too optimistic for the validation setting [18]. Calibration results were graphically displayed.

Analyses were stratified by geographic groups of studies, since stratification by individual studies would provide too few events in some strata [5, 13, 19]. To allow for heterogeneity across multiple studies, random-effect meta-analyses were performed. We calculated 95% confidence intervals (CI) and 95% prediction intervals (PI), which indicate the likely range for prediction accuracy of the model in a new dataset, for discrimination and calibration measures. A sensitivity analysis was performed to check the consistency of CBCrisk performance measures when metachronous CBC was defined as an event after 6 instead of 3 months since the first BC diagnosis. More details are provided in the Supplementary Material. All analyses were implemented using SAS (SAS Institute Inc., NC, USA) and R software [20].

Results

We included 132,756 patients from 20 studies who experienced 4862 CBC events during a median follow-up of 8.8 years. The main patient and clinical characteristics across studies and geographic areas are shown in Table 2.

Table 2.

Description of main patient and clinical factors used for evaluation of the models and formula

Studya/geographic area	Europe —otherb	Europe—Scandinavia	Europe—United Kingdom	Netherlands— BOSOM	Netherlands— EMC	Netherlands— NCR	United States and Australia
N	15,183	12,928	11,921	3760	3390	83,138	2436
Age at first diagnosis, years (%)
< 30	152 (1.0)	46 (0.4)	156 (1.3)	108 (2.9)	46 (1.4)	388 (0.5)	41 (1.7)
30–39	1252 (8.2)	489 (3.8)	1811 (15.2)	842 (22.4)	374 (11.0)	4241 (5.1)	494 (20.3)
40 +	13,779 (90.8)	12,393 (95.9)	9954 (83.5)	2810 (74.7)	2970 (87.6)	78,509 (94.4)	1901 (78.0)
Age at first birth = unknown (%)	15,183 (100.0)	12,928 (100.0)	11,921 (100.0)	3760 (100.0)	3390 (100.0)	83,138 (100.0)	2436 (100.0)
Family history (%)
Yes	2123 (14.0)	818 (6.3)	1371 (11.5)	737 (19.6)	591 (17.4)	0 (0.0)	319 (13.1)
No	8057 (53.1)	3158 (24.4)	8210 (68.9)	1177 (31.3)	2482 (73.2)	0 (0.0)	1498 (61.5)
Unknown	5003 (33.0)	8952 (69.2)	2340 (19.6)	1846 (49.1)	317 (9.4)	83,138 (100.0)	619 (25.4)
First BC type = Pure invasive (%)	15,183 (100.0)	12,928 (100.0)	11,921 (100.0)	3760 (100.0)	3390 (100.0)	83,138 (100.0)	2436 (100.0)
Breast density = unknown (%)	15,183 (100.0)	12,928 (100.0)	11,921 (100.0)	3760 (100.0)	3390 (100.0)	83,138 (100.0)	2436 (100.0)
ER status (%)
Negative	3387 (22.3)	1746 (13.5)	1718 (14.4)	896 (23.8)	842 (24.8)	14,591 (17.6)	445 (18.3)
Positive	10,071 (66.3)	9401 (72.7)	7175 (60.2)	2024 (53.8)	2427 (71.6)	64,790 (77.9)	1572 (64.5)
Unknown	1725 (11.4)	1781 (13.8)	3028 (25.4)	840 (22.3)	121 (3.6)	3757 (4.5)	419 (17.2)
High-risk pre-neoplasia = unknow n (%)	15,183 (100.0)	12,928 (100.0)	11,921 (100.0)	3760 (100.0)	3390 (100.0)	83,138 (100.0)	2436 (100.0)
Anti-estrogen therapy (%)
Yes	7868 (51.8)	6434 (49.8)	8712 (73.1)	809 (21.5)	1559 (46.0)	40,214 (48.4)	363 (14.9)
No	4570 (30.1)	1947 (15.1)	2046 (17.2)	2739 (72.8)	1821 (53.7)	42,924 (51.6)	8 (0.3)
Unknown	2745 (18.1)	4547 (35.2)	1163 (9.8)	212 (5.6)	10 (0.3)	0 (0.0)	2065 (84.8)
CBC cumulative incidence (%)
3-year (95% CI)	1.0 (0.8–1.2)	0.7 (0.5–0.9)	0.5 (0.3–0.7)	1.7 (1.3–2.1)	1.7 (1.2–2.1)	1.3 (1.2–1.4)	1.8 (0.8–2.8)
5-year (95% CI)	1.6 (1.4–1.9)	1.0 (0.8–1.3)	1.0 (0.8–1.3)	3.0 (2.5–3.6)	2.6 (2.1–3.2)	2.4 (2.3–2.5)	2.8 (1.7–3.8)
10-year (95% CI)	3.5 (3.1–3.9)	2.1 (1.7–2.4)	1.3 (1.0–1.5)	5.5 (4.7–6.2)	5.7 (4.9–6.6)	4.6 (4.5–4.8)	4.1 (3.0–5.3)

More details about the main patient and clinical characteristics by study are available in the supplementary information of [5]

BOSOM Breast Cancer Outcome Study of Mutation carriers, EMC Erasmus Medical Center, NCR Netherlands Cancer Registry, BC breast cancer, ER estrogen receptor, CBC contralateral breast cancer, CI confidence interval

^aThe studies denoted with Europe and United States and Australia are part of the Breast Cancer Association Consortium

^bEurope—other geographic area included studies from Belgium (1), Germany (2), Netherlands (2) and Poland (2)

The AUCs at 5 and 10 years was around 0.6: 0.59 (95% CI 0.57–0.61; 95% PI 0.54–0.64) and 0.58 (95% CI 0.57–0.59; 95% PI 0.55–0.61) for CBCrisk (Fig. 1); 0.61 (95% CI 0.60–0.62; 95% PI 0.59–0.63) and 0.60 (95% CI 0.59–0.61; 95% PI 0.58–0.62) for the Manchester formula (Fig. 2). The E/O ratio at 5 and 10 years was close to 1 for all models: 0.86 (95% CI 0.50–1.46; 95% PI 0.20–3.75) and 0.82 (95% CI 0.51–1.32; 95% PI 0.21–3.14) for CBCrisk (Table 3); 1.54 (95% CI 0.61–3.92; 95% PI 0.11–20.72, Table 4), and 1.53 (95% CI 0.63–3.73; 95% PI 0.13–18.52) for the Manchester formula (Table 4); 1.26 (95% CI 0.57–2.77; 95% PI 0.14–11.34), and 1.28 (95% CI 0.63–2.58; 95% PI 0.18–9.18) for PredictCBC-1A (Table 5); 1.33 (95% CI 0.59–2.99, 95% PI 0.14–12.76), 1.35 (95% CI 0.65–2.77; 95% PI 0.19–10.24) for PredictCBC-1B (Table 5) [5]. The calibration slope was close to 1 for CBCrisk (1.26, 95% CI 1.01–1.50 and 95% PI 1.01–1.50, Tables 3, 4, 5), and PredictCBC-1A and 1B 0.90 (95% CI 0.79–1.02; 95% PI 0.73–1.08), and 0.81 (95% CI 0.63–0.99; 95% PI 0.50–1.12) (Table 5), while prognostic effects were far too large for the Manchester formula (slope: 0.39, 95% CI 0.34–0.43, 95% PI 0.34–0.43, Tables 4, 5). Calibration plots of CBCrisk at 5 and 10 years are shown in Supplementary Fig. 1 and Supplementary Fig. 2. As reported previously [5], the AUCs at 5 and 10 years for PredictCBC-1A were 0.63 (95% CI 0.58–0.67, 95% PI 0.52–0.74), and 0.63 (95% CI 0.59–0.66, 95% PI 0.53–0.72), respectively; for PredictCBC-1B 0.59 (CI 0.54–0.63, 95% PI 0.46–0.71, Table 5), and 0.59 (95% CI 0.56–0.62, 95% PI 0.52–0.66, Table 5), respectively.

Fig. 1.

Prediction performance of the CBCrisk model (Chowdhury et al. [7]). The upper and lower panel show the discrimination assessed by a time-dependent Area-Under-the-Curve at 5 and 10 years, respectively. The black squares indicate the estimated accuracy of a model built on all remaining studies or geographic areas. The black horizontal lines indicate the corresponding 95% confidence intervals of the estimated accuracy (interval whiskers). The black diamonds indicate the mean with the corresponding 95% confidence interval of the predictive accuracy

Fig. 2.

Prediction performance of the Manchester formula (Basu et al. [8]) The upper and lower panel show the discrimination assessed by a time-dependent Area-Under-the-Curve at 5 and 10 years, respectively. The black squares for each dataset indicate the estimated accuracy of a model built on all remaining studies or geographic areas. The black horizontal lines indicate the corresponding 95% confidence intervals of the estimated accuracy (interval whiskers). The black diamonds indicate the mean with the corresponding 95% confidence interval of the predictive accuracy

Table 3.

Calibration performance of the CBC risk model

Validation dataset	E/O ratio at 5 years (95% CI)	E/O ratio at 10 years (95% CI)	Calibration slope (95% CI)
Europe—Other	0.87 (076 to 0.98)	0.75 (0.68 to 0.81)	1.11 ( 0.40 to 1.83)
Europe—Scandinavia	1.59 (1.28 to 1.91)	1.23 (1.08 to 1.38)	0.86 ( 0.16 to 1.57)
Europe—UK	1.35 (1.38 to 2.17)	1.82 (1.53 to 2.11)	0.85 (− 0.03 to 1.73)
Netherlands—BOSOM	0.45 (0.37 to 0.53)	0.50 (0.43 to 0.57)	1.34 ( 0.76 to 1.93)
Netherlands—EMC	0.48 (0.38 to 0.57)	0.43 (0.37 to 0.50)	1.19 ( 0.65 to 1.73)
Netherlands—NCR	0.57 (0.54 to 0.59)	0.54 (0.52 to 0.56)	1.40 ( 1.11 to 1.68)
US and Australia	0.43 (0.33 to 0.54)	0.56 (0.45 to 0.67)	1.13 ( 0.25 to 2.00)
Meta-analysis	0.86 (0.50 to 1.46)	0.82 (0.51 to 1.32)	1.26 ( 1.01 to 1.50)
95% PI	0.20 to 3.75	0.21 to 3.14	1.01 to 1.50

Chowdhury et al. [7]

E/O expected-observed, CI confidence interval, UK United Kingdom, BOSOM Breast Cancer Outcome Study of Mutation carriers, EMC Erasmus Medical Center, NCR Netherlands Cancer Registry, PI prediction interval

Table 4.

Calibration performance of the Manchester formula

Validation dataset	E/O ratio at 5 years (95% CI)	E/O ratio at 10 years (95% CI)	Calibration slope (95% CI)
Europe—Other	1.64 (1.44 to 1.85)	1.46 (1.34 to 1.58)	0.40 (0.29 to 0.50)
Europe—Scandinavia	2.61 (2.09 to 3.12)	2.11 (1.85 to 2.37)	0.35 (0.13 to 0.57)
Europe—UK	3.34 (2.60 to 4.08)	3.49 (2.93 to 4.05)	0.42 (0.23 to 0.61)
Netherlands—BOSOM	0.81 (0.66 to 0.96)	0.92 (0.79 to 1.05)	0.45 (0.33 to 0.56)
Netherlands—EMC	0.94 (0.75 to 1.14)	0.87 (0.75 to 1.00)	0.35 (0.21 to 0.49)
Netherlands—NCR	1.00 (0.95 to 1.04)	1.01 (0.98 to 1.05)	0.37 (0.33 to 0.42)
US and Australia	0.77 (0.58 to 0.96)	1.02 (0.82 to 1.23)	0.51 (0.33 to 0.68)
Meta-analysis	1.54 (0.61 to 3.92)	1.53 (0.63 to 3.73)	0.39 (0.34 to 0.43)
95% PI	0.11 to 20.72	0.13 to 18.52	0.34 to 0.43

Basu et al. [8]

E/O expected-observed, CI confidence interval, UK United Kingdom, BOSOM Breast Cancer Outcome Study of Mutation carriers, EMC Erasmus Medical Center, NCR Netherlands Cancer Registry, PI prediction interval

Table 5.

Summary of prediction performance of CBCrisk, Manchester formula, and PredictCBC version 1A and version 1B with the corresponding 95% prediction intervals (PI)

Characteristics	CBCriska	Manchester formulab	PredictCBC version 1Ac,d	PredictCBC version 1Bc,d
Discrimination
AUC at 5 years (95% PI)	0.59 (0.54 to 0.64)	0.61 (0.59 to 0.63)	0.63 (0.52 to 0.74)	0.59 (0.46 to 0.71)
AUC at 10 years (95% PI)	0.58 (0.55 to 0.61)	0.60 (0.58 to 0.62)	0.63 (0.53 to 0.72)	0.59 (0.52 to 0.66)
Calibration
E/O ratio at 5 years (95% PI)	0.86 (0.20 to 3.75)	1.54 (0.11 to 20.72)	1.26 (0.14 to 11.34)	1.33 (0.14 to 12.76)
E/O ratio at 10 years (95% PI)	0.82 (0.21 to 3.14)	1.53 (0.13 to 18.52)	1.28 (0.18 to 9.18)	1.35 (0.19 to 10.24)
Slope (95% PI)	1.26 (1.01 to 1.50)	0.39 (0.34 to 0.43)	0.90 (0.73 to 1.08)	0.81 (0.50 to 1.12)

AUC Area under the curve, PI prediction interval

^aChowdhury et al. [7]

^bBasu et al. [8]

^cGiardiello et al. [5], Fig. 1 and Figure S5

^dversion 1A includes BRCA mutation status as a variable while 1B does not

Sensitivity analysis showed that the performance measures of CBCrisk did not change when metachronous CBC was defined after 6 months since first BC diagnosis (see Supplementary Materials, Supplementary Tables 1, 2 and Supplementary Fig. 3).

Discussion

Accurate CBC risk predictions are essential in clinical decision-making around CPM or tailored surveillance among patients with first primary BC. In particular, overestimation of risk can lead to recommending CPM among BC patients with low risks. Underestimation can lead to suboptimal surveillance or hesitance about recommending CPM for patients with substantial risk. Using individual patient data from multiple studies with long follow-up, we externally evaluated the prediction performance accuracy of CBCrisk, a tool developed and validated to provide individualized CBC risk prediction, and the Manchester formula, a heuristically derived calculation of CBC lifetime risk [6–9]. In addition, the availability of different European-descendent studies allowed heterogeneity in the performance by geographic area to be assessed.

CBCrisk under-predicted the risk of CBC and had moderate discrimination ability with considerable heterogeneity between studies. The Manchester formula was empirically derived from a systematic review, and its discrimination accuracy was higher than CBCrisk. This may be explained by the inclusion of BRCA1/2 mutation carrier information, an important determinant of CBC risk [21]. With the same large individual patient data sets, PredictCBC models had been developed and validated [5]. In particular, PredictCBC version 1A includes information of BRCA1/2 mutation carriers and extensive information about the primary BC including treatments. The discrimination of all three prediction models was moderate, with AUC values around 0.6.

CBCrisk was previously externally validated using two independent clinical studies from Johns Hopkins University (JH) and MD Anderson Cancer Center (MDA) in the US [9]. Discrimination ability was 0.61 and 0.65 at 3 years, and 0.62 and 0.61 at 5 years for JH and MDA, respectively. The risk of CBC was overestimated in JH with E/O ratios of 2.02 and 1.56 at 3 and 5 years, while underestimated in MDA with E/O ratios of 0.61 and 0.62, respectively.

The considerable heterogeneity in all CBC risk calculators, especially in the CBCrisk and the Manchester formula, reflects the different CBC incidences in every study [13]. Another potential source of heterogeneity is the carrier frequency of germline mutations associated with CBC that may vary among studies, especially in the CBC calculators not including information of BRCA1/2 mutation as CBCrisk and the PredictCBC-1B [22]. In addition, heterogeneity may be due to the different proportions of the use of (neo)adjuvant systemic therapies explained by the different distribution of tumor subtypes among studies [4]. Besides, inter-observer variation in pathological examination of BC among studies may lead to different adjuvant systemic therapy advice and, consequently, prediction of CBC risk [23]. Variation in prediction performance and limited generalizability of CBC risk calculators can also be partially explained by differences in how predictors are measured among studies [24, 25]. For example, lack of family history knowledge may lead to uncertainty in risk prediction and varies according to demographics of the patients [26]. In particular, if in some studies BC patients misreported information about family history, the CBC risk would be over(under)estimated causing inappropriate decision-making regarding CPM or tailored surveillance. Some limitations of our study must be recognized. First, our dataset, while large, had missing data for three covariates that were used in the CBCrisk model: breast density, age at first birth, and high-risk pre-neoplasia. The authors of CBCrisk estimated the relative risks for patients with the unknown characteristics, but the use of the missing indicator variable is suboptimal compared to having the prognostic information available. It may lead to over or under-estimation of absolute CBC risk [27]. For this reason, we suggest that it is preferable to use multiple imputation of missing data, as is done in the PredictCBC models [28, 29]. In addition, investigation of the potential source of model misspecification due to possible different definitions or measurement error was not possible [30–32].

In conclusion, current statistical risk prediction models and heuristic formulas provided moderate CBC individualized prediction performance. Careful re-calibration is required before considering these models for clinical decision-making. A more direct comparison between the current CBC risk prediction models using a large external dataset with complete information on all factors included in all CBC prediction models would be ideal, but is currently unavailable. There is an ongoing debate about improvements of clinical prediction performance using machine learning approaches compared to standard regression approaches for risk prediction [33, 34]. However, irrespective of the methodology, better predictors are needed to predict CBC more accurately. Deeper biological insights and potential inclusion of other genetic markers such as CHEK2 c.1100del mutation status and polygenic risk scores based on common genetic variants may improve CBC risk prediction, although rare mutations are unlikely to contribute substantially to CBC risk in the general population [35, 36]. Life-style factors such as body mass index, alcohol consumption, and smoking also may help to better stratify high- and low-CBC risk patients even though these factors are difficult to measure accurately. Moreover, breast density may be important. More detailed information about adjuvant systemic therapies may better identify patients with low- and high-CBC risk since chemotherapy and especially endocrine therapy reduce CBC risk [4]. After extension and further external validation of prediction models for CBC risk, investigation of their potential clinical utility is an important future step.