Accuracy of Risk Estimates from the iPrevent Breast Cancer Risk Assessment and Management Tool

Abstract

Background

iPrevent is an online breast cancer (BC) risk management decision support tool. It uses an internal switching algorithm, based on a woman’s risk factor data, to estimate her absolute BC risk using either the International Breast Cancer Intervention Study (IBIS) version 7.02, or Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm version 3 models, and then provides tailored risk management information. This study assessed the accuracy of the 10-year risk estimates using prospective data.

Methods

iPrevent-assigned 10-year invasive BC risk was calculated for 15 732 women aged 20–70 years and without BC at recruitment to the Prospective Family Study Cohort. Calibration, the ratio of the expected (E) number of BCs to the observed (O) number and discriminatory accuracy were assessed.

Results

During the 10 years of follow-up, 619 women (3.9%) developed BC compared with 702 expected (E/O = 1.13; 95% confidence interval [CI] =1.05 to 1.23). For women younger than 50 years, 50 years and older, and BRCA1/2-mutation carriers and noncarriers, E/O was 1.04 (95% CI = 0.93 to 1.16), 1.24 (95% CI = 1.11 to 1.39), 1.13 (95% CI = 0.96 to 1.34), and 1.13 (95% CI = 1.04 to 1.24), respectively. The C-statistic was 0.70 (95% CI = 0.68 to 0.73) overall and 0.74 (95% CI = 0.71 to 0.77), 0.63 (95% CI = 0.59 to 0.66), 0.59 (95% CI = 0.53 to 0.64), and 0.65 (95% CI = 0.63 to 0.68), respectively, for the subgroups above. Applying the newer IBIS version 8.0b in the iPrevent switching algorithm improved calibration overall (E/O = 1.06, 95% CI = 0.98 to 1.15) and in all subgroups, without changing discriminatory accuracy.

Conclusions

For 10-year BC risk, iPrevent had good discriminatory accuracy overall and was well calibrated for women aged younger than 50 years. Calibration may be improved in the future by incorporating IBIS version 8.0b.

Clinical decision support tools can help integrate and personalize evidence to assist clinicians and patients to make better-informed decisions (1). Tools that are computerized, personalized, and accessible not only to clinicians but also to patients themselves have better uptake and result in greater adherence to recommended practice (2). iPrevent is a new, evidence-based, personalized breast cancer (BC) risk assessment and risk management decision support tool (3) developed to assess risk across the entire risk spectrum. It is freely available online at https://www.petermac.org/iprevent. It is designed for collaborative use between women and their clinicians and was developed to be consistent with best practice in risk communication. iPrevent assesses a woman’s personal BC risk and provides tailored risk management advice, including information about chemoprevention, risk-reducing surgeries, lifestyle modification, and screening (Figure 1) (4).

Figure 1.

Screenshots of the Online iPrevent Breast Cancer Risk Assessment and Risk Management Decision Support Tool. A) iPrevent gathers information about lifestyle and medical and family history and, using that information, provides 10-year breast cancer risk estimates as well as lifetime risk estimates (not shown). B) iPrevent provides a menu of appropriate risk management options for each woman depending on her category of risk, based on her absolute risk estimate and Cancer Australia guidelines. An example for a woman at moderate risk of breast cancer. C) iPrevent provides personalized estimates of the absolute reduction in breast cancer risk that may be expected with prevention strategies such as tamoxifen.

There are many available BC-risk prediction models, but their use in the clinic can be hampered by a lack of clear guidelines on which models should be applied in which circumstances. iPrevent chooses from two commonly used pedigree-based models, according to the algorithm shown in Figure 2—the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA) version 3 (v3) (5) or the International Breast Cancer Intervention Study (IBIS) model version 7.02 (v7.02) (6)—to estimate each woman’s 10-year and lifetime BC risk based on her risk factors. Both BOADICEA and IBIS, which we have recently shown (7) are well calibrated and outperform other commonly used risk models like Gail (8) and BRCAPRO (9), are applicable to all women across the spectrum of absolute familial risk. In addition, IBIS incorporates nonfamilial risk factors and is appropriate for women without a BC family history. Other commonly used BC models such as the Gail (8) or Breast Cancer Surveillance Consortium model (10) have been developed and validated only for women at average risk.

Figure 2.

iPrevent software algorithm for breast cancer risk estimation. Based on their personal and/or first- or second-degree family history of cancer and BRCA1/BRCA2-mutation status, the iPrevent 10-year breast cancer risk estimate was derived from the IBIS model for 9749 participants and the BOADICEA model for the remaining 5983 participants. # IBIS- or BOADICEA-derived risk estimates were further modified to obtain the iPrevent risk estimate by applying a 33% relative risk reduction for women who had taken tamoxifen for breast cancer prevention prior to baseline and a 50% relative risk reduction for women who had had a bilateral oophorectomy before age 45 years. AH = atypical hyperplasia; BOADICEA = Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm; BRCA mutation = BRCA1 or BRCA2 pathogenic germline mutation; IBIS = International Breast Cancer Intervention Study; LCIS = lobular carcinoma in situ.

After providing a personal BC risk estimate, iPrevent gives a menu of appropriate potential risk management interventions based on Australian National Guidelines (11,12), applying published relative risk reductions for relevant interventions (eg, 33% relative risk reduction for 5 years of tamoxifen use) (13) to give personalized, detailed, quantitative information on how the intervention changes a woman’s absolute BC risk. Women and their doctors can choose to see this information presented as statistics, pictograms, and/or graphs. Individualized lifestyle advice is also provided, based on the risk factor data the woman supplies.

A pilot study of iPrevent use by women and their clinicians found it had good usability and acceptability and suggested that it might improve the accuracy of BC risk perception and enhance BC prevention knowledge without increasing BC worry or anxiety (14).

The primary aim of this study was to assess the calibration and discrimination of the 10-year BC risk estimates provided by the current online version of iPrevent, using data from a prospective cohort. During the course of this study, v8.0b of IBIS was released, which updated model parameters for the association between hormone replacement therapy (HRT) and BC risk and also allowed for optional inclusion of mammographic density and single nucleotide polymorphism (SNP) data. Hence, we also prospectively validated the iPrevent switching algorithm, incorporating IBIS v8.0b.

Methods

This validation study was performed using data from the Prospective Family Study Cohort (15), which pooled data from two cohorts: the Breast Cancer Family Registry (BCFR) (16) and the Kathleen Cuningham Foundation Consortium for Research into Familial Breast Cancer (kConFab) Follow-Up Study (17). These cohorts are enriched for women with an increased familial risk of BC. The BCFR and kConFab used the same risk factor questionnaire at enrollment, which collected data on demographics, height, weight, breast and ovarian surgeries, reproductive history, and lifestyle factors. The cancer family history questions asked about breast and other cancers diagnosed in participants and their first- and second-degree relatives, including ages at diagnosis and age last known to be alive, for all family members. Screening for germline BRCA1 and BRCA2 mutations by the BCFR and kConFab typically involved testing the affected proband and/or the youngest affected woman in each family, followed by cascade testing of other family members for the family-specific mutation if present in the proband. Incident BC diagnoses were verified through pathology reports for 84% of cases. Participants provided written informed consent and were followed for a median of 10 years. The cohort protocols were approved by institutional review boards at each of the respective institutions.

At baseline, there were 18 856 women in the Prospective Family Study Cohort without a personal history of invasive or in situ BC (15). We excluded from the analyses those women who had bilateral mastectomy prior to cohort entry (n = 113), those with follow-up time of less than 2 months (n = 517), those with prior history of ovarian cancer (n = 316), those younger than 20 years or older than 70 years at baseline (n = 2082), and those for whom inadequate pedigree data were available (n = 96), leaving 15 732 eligible participants from 6694 families.

We manually applied the iPrevent switching algorithm to the entire cohort (which allowed us to consider the impact of two versions of IBIS: 7.02 and 8.0b). We first determined for each participant whether iPrevent would use the IBIS or BOADICEA model to generate the iPrevent risk estimate (Figure 2). The software packages IBIS v7.02 (http://www.ems-trials.org/riskevaluator/) and BOADICEA v3 (https://pluto.srl.cam.ac.uk/cgi-bin/bd3/v3/bd.cgi) were then used to assign 10-year BC risks. These risk estimates were then further modified based on preventive measures used prior to baseline by applying a 33% reduction in relative risk for women who had taken tamoxifen for BC prevention prior to baseline and a 50% reduction in relative risk for women who had had a bilateral oophorectomy before age 45 years (although the validity of the latter has recently been questioned [18], and this may need future modification). The same process was subsequently undertaken using IBIS v8.0b, but mammographic density and SNP-based polygenic risk (which can be entered in IBIS v8.0b but not v7.2) were not available for this dataset (19).

Statistical Analysis

The calibration and discriminatory accuracy of the 10-year iPrevent BC risk estimates were assessed for the overall cohort and also for subgroups 1) younger than 50 years at baseline, 2) 50 years or older at baseline, 3) BRCA1- and BRCA2- mutation carriers (combined), and 4) noncarriers. Follow-up time was censored at date of in situ BC diagnosis, bilateral mastectomy, death, loss to follow-up, or on reaching 10 years. If follow-up time was less than 10 years, we used the method described by Amir and colleagues (20) to adjust the predicted risk to the available follow-up time.

For calibration, the expected number of invasive BC cases based on the iPrevent-assigned risk (E) was compared with the observed number of cases (O) (21). The 95% confidence intervals (CI) for the ratio of expected to observed numbers (E/O) was calculated by E/O*EXP(± 1.96*SQRT[1/O]) (22). For the overall sample and in the younger than age 50 years, 50 years and older, and noncarrier subgroups, the mean iPrevent-assigned risk to observed BC incidence was calculated overall and for each decile of risk, with separate deciles used for BRCA1- and BRCA2-mutation carriers. To compare calibration between IBIS v7.02 and v8.0b, we calculated the estimated calibration index (ECI) defined as the average squared difference between observed and predicted risk. Values of ECI closer to zero indicate better calibration (23).

To evaluate discriminatory accuracy, the receiver operating characteristic curves for the development of invasive BC within 10 years, overall, and for the subgroups defined above were plotted and the C-statistic computed. A C-statistic of 0.6–0.7 is considered “sufficient,” 0.7–0.8 is considered “good,” and 0.8–0.9 “very good” (21). The calibration and discrimination analyses were conducted using the RMAP 0.03–01 (https://gailg.github.io/rmap/) algorithm in the R environment and Stata version 14 (Stata Corporation, College Station, TX). All other analyses were conducted using SAS version 9.4.

Results

Of the 15 732 eligible women, 619 (3.9%) were diagnosed with invasive BC and 415 (2.6%) died without a BC diagnosis during the 10 years of follow-up. An additional 6078 were alive and without invasive BC with less than 10 years' follow-up (and were censored at date of last follow-up), leaving 8620 women unaffected with invasive BC at 10 years. Table 1 shows the distribution of risk factors for each outcome group. Most women (62.3%) were younger than 50 years at baseline. There were 584 BRCA1-mutation carriers (3.7%) and 491 BRCA2-mutation carriers (3.1%) identified in the study sample. Of the total sample, 21% reported use of HRT.

Table 1.

Distribution of risk factors by breast cancer outcome

Risk factors	Unaffected after 10 y (n = 8620) No. (%)	Follow-up <10 y (n = 6078) No. (%)	Died within 10 y (n = 415) No. (%)	Breast cancer within 10 y (n = 619) No. (%)	All (n = 15 732) No. (%)	P
Age at interview, y						<.001
20–29	1197 (13.9)	1053 (17.3)	5 (1.2)	34 (5.5)	2289 (14.5)
30–39	2004 (23.2)	1383 (22.8)	24 (5.8)	119 (19.2)	3530 (22.4)
40–49	2239 (26.0)	1508 (24.8)	55 (13.3)	177 (28.6)	3979 (25.3)
50–59	1877 (21.8)	1292 (21.3)	117 (28.2)	162 (26.2)	3448 (21.9)
60–70	1303 (15.1)	842 (13.9)	214 (51.6)	127 (20.5)	2486 (15.8)
Race/ethnicity						<.001
Non-Hispanic white	7390 (85.7)	4281 (70.4)	318 (76.6)	522 (84.3)	12 511 (79.5)
Non-Hispanic black	236 (2.7)	448 (7.4)	38 (9.2)	16 (2.6)	738 (4.7)
Hispanic	309 (3.6)	905 (14.9)	34 (8.2)	38 (6.1)	1286 (8.2)
Asian	336 (3.9)	223 (3.7)	9 (2.2)	26 (4.2)	594 (3.8)
Other	251 (2.9)	163 (2.7)	11 (2.7)	11 (1.8)	436 (2.8)
Unknown	98 (1.1)	58 (1.0)	5 (1.2)	6 (1.0)	167 (1.1)
Age at menarche, y						<.01
≤11	1466 (17.0)	1091 (17.9)	83 (20.0)	91 (14.7)	2731 (17.4)
12–13	4543 (52.7)	3069 (50.5)	188 (45.3)	331 (53.5)	8131 (51.7)
≥ 14	2530 (29.4)	1823 (30.0)	140 (33.7)	190 (30.7)	4683 (29.8)
Unknown	81 (0.9)	95 (1.6)	4 (1.0)	7 (1.1)	187 (1.2)
BMI, kg/m2						<.001
< 25	4830 (56.0)	2924 (48.1)	174 (41.9)	309 (49.9)	8237 (52.4)
25–<30	2213 (25.7)	1617 (26.6)	125 (30.1)	177 (28.6)	4132 (26.3)
≥ 30	1402 (16.3)	1413 (23.2)	103 (24.8)	128 (20.7)	3046 (19.4)
Unknown	175 (2.0)	124 (2.0)	13 (3.1)	5 (0.8)	317 (2.0)
Age at first live birth, y						<.001
< 20	993 (11.5)	913 (15.0)	96 (23.1)	71 (11.5)	2073 (13.2)
20–24	2520 (29.2)	1675 (27.6)	167 (40.2)	206 (33.3)	4568 (29.0)
25–29	1953 (22.7)	1188 (19.5)	64 (15.4)	121 (19.5)	3326 (21.1)
≥ 30	922 (10.7)	642 (10.6)	29 (7.0)	94 (15.2)	1687 (10.7)
Nulliparous	2232 (25.9)	1660 (27.3)	59 (14.2)	127 (20.5)	4078 (25.9)
Hormones taken for menopause						<.001
Ever	1930 (22.4)	1137 (18.7)	169 (40.7)	161 (26.0)	3397 (21.6)
Never	6438 (74.7)	4827 (79.4)	235 (56.6)	436 (70.4)	11 936 (75.9)
Unknown	252 (2.9)	114 (1.9)	11 (2.7)	22 (3.6)	399 (2.5)
Menopausal status						<.001
Pre-	4946 (57.4)	3730 (61.4)	66 (15.9)	298 (48.1)	9040 (57.5)
Post-	2855 (33.1)	1998 (32.9)	326 (78.6)	268 (43.3)	5447 (34.6)
Unknown	819 (9.5)	350 (5.8)	23 (5.5)	53 (8.6)	1245 (7.9)
Age at menopause (among postmenopausal), y						<.001
<40	334 (11.7)	259 (13.0)	41 (12.6)	37 (13.8)	671 (12.3)
40–49	935 (32.7)	671 (33.6)	102 (31.3)	84 (31.3)	1792 (32.9)
≥50	1027 (36.0)	704 (35.2)	119 (36.5)	111 (41.4)	1961 (36.0)
Unknown	559 (19.6)	364 (18.2)	64 (19.6)	36 (13.4)	1023 (18.8)
Benign breast disease						<.001
Yes	2353 (27.3)	1623 (26.7)	105 (25.3)	219 (35.4)	4300 (27.3)
No	5972 (69.3)	4330 (71.2)	302 (72.8)	377 (60.9)	10 981 (69.8)
Unknown	295 (3.4)	125 (2.1)	8 (1.9)	23 (3.7)	451 (2.9)
BRCA1/2 mutation						<.0001
BRCA1 carrier	184 (2.1)	301 (5.0)	17 (4.1)	82 (13.2)	584 (3.7)
BRCA2 carrier	129 (1.5)	294 (4.8)	13 (3.1)	55 (8.9)	491 (3.1)
Noncarrier*	8307 (96.4)	5483 (90.2)	385 (92.8)	482 (77.9)	14 657 (93.2)
Number of first-degree relatives with breast cancer						<.001†
0	1477 (17.1)	1157 (19.0)	88 (21.2)	83 (13.4)	2805 (17.8)
1	5633 (65.3)	3878 (63.8)	232 (55.9)	347 (56.1)	10 090 (64.1)
≥2	1510 (17.5)	1043 (17.2)	95 (22.9)	189 (30.5)	2837 (18.0)
Number of second-degree relatives with breast cancer						<.001†
0	4173 (48.4)	2678 (44.1)	203 (48.9)	240 (38.8)	7294 (46.4)
1	2858 (33.2)	2042 (33.6)	139 (33.5)	227 (36.7)	5266 (33.5)
≥2	1589 (18.4)	1358 (22.3)	73 (17.6)	152 (24.6)	3172 (20.2)
Number of third-degree relatives with breast cancer						<.001†
0	5794 (67.2)	3701 (60.9)	291 (70.1)	386 (62.4)	10 172 (64.7)
1	1673 (19.4)	1227 (20.2)	57 (13.7)	127 (20.5)	3084 (19.6)
≥2	1153 (13.4)	1150 (18.9)	67 (16.1)	106 (17.1)	2476 (15.7)

^*Noncarriers are defined as women not known to be BRCA1- or BRCA2-mutation carriers, so they include tested and untested women. BMI = body mass index.

^† P value was also <.001 after adjusting for age at baseline.

The calibration results are summarized in Table 2. The iPrevent-assigned number of expected incident BCs was 702 compared with 619 observed (E/O = 1.13, 95% CI = 1.05 to 1.23). For women younger than 50 years at baseline, there were 343 expected cases of BC and 330 observed (E/O = 1.04, 95% CI = 0.93 to 1.16). For women 50 years and older at baseline, there were 359 expected cases of BC and 289 observed (E/O = 1.24, 95% CI = 1.11 to 1.39). For BRCA1- or BRCA2-mutation carriers, there were 155 expected cases of BC compared with 137 observed (E/O = 1.13, 95% CI = 0.96 to 1.34) and for noncarriers, 547 expected and 482 observed (E/O = 1.13, 95% CI = 1.04 to 1.24). A sensitivity analysis excluding 407 women who had taken tamoxifen or been on a chemoprevention trial prior to their BC diagnosis did not appreciably change these results (data not shown).

Table 2.

Calibration and discrimination of iPrevent overall and by subgroups^*

	Current iPrevent (BOADICEA version 3and IBIS version 7.02)					Future iPrevent (BOADICEA version 3and IBIS version 8.0b)
Subgroups	Expected	Observed	Expected/Observed (95% CI)	C-statistic (95% CI)	ECI	Expected	Observed	Expected/Observed (95% CI)	C-statistic (95% CI)	ECI
Overall	702	619	1.13 (1.05 to 1.23)	0.70 (0.68 to 0.73)	0.102	657	619	1.06 (0.98 to 1.15)	0.70 (0.68 to 0.73)	0.077
<50 y	343	330	1.04 (0.93 to 1.16)	0.74 (0.71 to 0.77)	0.076	332	330	1.01 (0.90 to 1.12)	0.74 (0.71 to 0.77)	0.062
≥50 y	359	289	1.24 (1.11 to 1.39)	0.63 (0.59 to 0.66)	0.179	325	289	1.13 (1.00 to 1.26)	0.63 (0.60 to 0.66)	0.117
BRCA1/2 carrier	155	137	1.13 (0.96 to 1.34)	0.59 (0.53 to 0.64)	0.982	155	137	1.13 (0.96 to 1.34)	0.59 (0.53 to 0.64)	0.982
Noncarrier	547	482	1.13 (1.04 to 1.24)	0.65 (0.63 to 0.68)	0.049	502	482	1.04 (0.95 to 1.14)	0.65 (0.63 to 0.68)	0.017

^*BOADICEA = Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm; ECI = estimated calibration index; IBIS = International Breast Cancer Intervention Study.

Figure 3A shows that, for the overall cohort, iPrevent was well calibrated overall except for those in the highest decile of risk, where iPrevent overpredicted risk (mean 10-year iPrevent-expected risk of 19.1% compared with a mean observed risk of 14.7%). There were similar findings for all subgroups except for women younger than 50 years at baseline, where calibration was good across the spectrum of risk (Figure 3, C, E, G, and I), including the highest-risk decile.

Figure 3.

Calibration of iPrevent 10-year estimates of invasive breast cancer for women in the Prospective Family Study Cohort by quantile, overall, and for subgroups by age and mutation carriers. Calibration of iPrevent estimated 10-year breast cancer risk by decile. The coordinates on the x-axis represent the mean 10-year expected risks from iPrevent. The coordinates on the y-axis represent the estimates of 10-year breast cancer probabilities based on the women’s observed breast cancer status, and the bars denote 95% confidence intervals for the observed risk. For each subgroup, calibration was assessed using BOADICEA V3 and IBIS V7.02 to emulate the current online version of iPrevent (A, C, E, G, I) and using BOADICEA V3 and IBIS V8.0b to simulate a future updated version of iPrevent (B, D, F, H, J). BOADICEA = Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm; IBIS = International Breast Cancer Intervention Study.

When the analyses were repeated using the same switching algorithm for iPrevent (Figure 2) but with IBIS v8.0b instead of IBIS v7.02, the calibration improved overall (E/O = 1.06, 95% CI = 0.98 to 1.15; Table 2) and for noncarriers and both age subgroups, but particularly for women 50 years and older (lower ECIs; Table 2). Calibration for BRCA1- and BRCA2-mutation carriers was unchanged because the iPrevent switching algorithm is constrained to choose BOADICEA as the risk estimation model for mutation carriers (Table 2 and Figure 3 B, D, F, H, and J). Regarding discriminatory accuracy, the C-statistic for the entire sample was 0.70 (95% CI = 0.68 to 0.73), 0.74 (95% CI = 0.71 to 0.77) for women younger than 50 years at baseline, 0.63 (95% CI = 0.59 to 0.66) for those 50 years and older at baseline, and 0.59 (95% CI = 0.53 to 0.64) for BRCA1- and BRCA2-mutation carriers (combined) and 0.65 (95% CI = 0.63 to 0.68) for noncarriers (Figure 4 and Table 2). The discriminatory accuracy did not change appreciably when analyses were repeated using the same switching algorithm for iPrevent (Figure 2), but incorporating IBIS v8.0b instead of IBIS v7.02 (Table 2).

Figure 4.

Receiver operating characteristic curves for iPrevent 10-year estimates of invasive breast cancer for women in the Prospective Family Study Cohort overall and by age and mutation status. BOADICEA = Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm; IBIS = International Breast Cancer Intervention Study.

Discussion

Accurate estimation of a woman’s personal BC risk facilitates the use of evidence-based management strategies appropriate for her risk level and allows calculation of the absolute risk-reduction benefit from preventive interventions.

In this validation study, based on 10-year BC risk estimates, iPrevent was well calibrated for women younger than 50 years at baseline. This is an important group of women, for whom other relatively user-friendly risk estimation tools, such as the Gail model, do not perform well (8). The calibration of iPrevent for women 50 years or older and for BRCA1- and BRCA2-mutation carriers was generally good, except for those in the top decile of risk, where it tended to be overestimated. Specifically, iPrevent tended to overestimate risk for women 50 years and older with an assigned 10-year risk no less than 14.3% and for BRCA1- and BRCA2-mutation carriers with an assigned 10-year risk no less than 35.7%. The overestimation of risk found in this study for the highest-risk decile is consistent with our previous study, using the same sample, which found similar findings for the IBIS and BOADICEA models when used separately (7). Uptake of chemoprevention after baseline does not appear to explain the overestimation of risk; in our sensitivity analyses that excluded women who took chemoprevention medications prior to their first BC, the overall inferences were the same. Similarly, we do not believe the overestimation is explained by uptake of bilateral oophorectomy after baseline because we have recently shown that premenopausal bilateral oophorectomy did not reduce BC risk in this cohort (18). We also censored women at bilateral mastectomy. Regardless, the extent of overestimation is unlikely to be of clinical importance because the actual 10-year BC risks for these women substantially exceed thresholds for intensified screening and medical prevention (and for mutation carriers, risk-reducing mastectomy). Therefore, the overestimation would be unlikely to lead to an inappropriate change in their clinical management.

Importantly, when we used the iPrevent switching algorithm with IBIS v8.0b (rather than v7.02), iPrevent was well calibrated overall for all subgroups. Given that mammographic density and SNP data were not available for this validation study, the improved calibration, seen predominantly for women 50 years and older, suggests that the improved performance might be explained at least in part by the different treatment of HRT model parameters between the two versions of IBIS. HRT use has declined dramatically since the 2002 release of the Women’s Health Initiative findings (24), and the improvement in calibration using IBIS version 8.0b rather than IBIS v7.02 might be less relevant to a contemporary sample of women who may be less likely to use HRT (the vast majority of the baseline data in our study were collected prior to 2002). Nevertheless, using either version of IBIS, the iPrevent algorithm provides good calibration for women younger than 50 years.

iPrevent was found to have good discriminatory accuracy (C-statistic = 0.70, 95% CI = 0.68 to 0.73) overall and with C-statistics ranging from 0.59 to 0.74 for the subgroups. The overall discriminatory accuracy of iPrevent compares favorably with those found in our prior study of the Gail, BRCAPRO, BOADICEA v3, and IBIS v8.0b risk estimation models, based on the same sample. In that prior study, the C-statistics for the overall sample ranged between 0.60 (95% CI = 0.58 to 0.62) for the Gail model and 0.71 (95% CI = 0.69 to 0.73) for the IBIS v8.0b model, with a C-statistic of 0.70 (95% CI = 0.68 to 0.72) for BOADICEA v3 (7). However, the similar C-statistics for iPrevent, IBIS, and BOADICEA imply that the internal model-switching algorithm programmed into iPrevent does not substantially improve the discriminatory accuracy of the risk estimates provided by either model alone. Nevertheless, iPrevent has the advantage of a user interface that has been designed for use both by consumers and clinicians, and iPrevent provides tailored risk management advice, which the other tools do not provide.

iPrevent was developed in response to qualitative research conducted with Australian clinicians and women (25–27). They indicated that an evidence-based, computerized tool to facilitate collaborative assessment and management of BC risk by women and their health-care providers would address a major area of need (25–27), especially if the tool could be used by women and printed out prior to the consultation (to save time). iPrevent currently uses Australian guidelines to determine the risk management information that is provided to women at each risk level, but future iterations of iPrevent could be made country specific with modifications to align the risk management output with local guidelines (11,12,28–30).

A recent review identified 16 published BC risk estimation models (31), including IBIS and BOADICEA, but few of those models couple risk management to risk estimation, and studies have demonstrated that clinicians have difficulty applying existing risk models in clinical practice (32). There are some online risk management tools that enable users to receive information about their risk with suggestions on how to modify it, such as Cancer Australia's Familial Risk Assessment–Breast and Ovarian Cancer tool (33), the Pink Hope Know Your Risk tool (34), and the Washington University Your Disease Risk tool based on the Nurses’ Health Study (35). However, these models have either not used pedigree models and therefore are not applicable across the spectrum of risk (35) and/or the underlying risk estimation models that the tools use are not published, and there is no information available regarding the accuracy of the risk estimates they provide (33,34).

Strengths of this validation study include its large sample size with comprehensive and systematically collected baseline risk factor assessment and long follow-up (average more than 10 years), resulting in a large number of incident BCs. A limitation of our study was that most participants were non-Hispanic white. The IBIS and BOADICEA models that underpin the iPrevent risk prediction do not account for differences in underlying BC incidence rates by race or ethnicity, so further validation using racially diverse cohorts is important. Another potential limitation is that almost all women in our study had at least some family history of BC, so further validation using a cohort of women without a family history of the disease would be valuable. In addition, our study sample lacked data on mammographic density and SNP-based polygenic risk, which are included in IBIS v8.0b and have been shown to enhance BC risk prediction (36,37), thus our study may underestimate the accuracy of BC risk estimates that would be provided if IBIS v8.0b were incorporated into iPrevent in the future.

A key advantage of iPrevent is that it can easily be used to apply relative risk for preventive factors through multiplying the absolute risks from the base models by the relative risk, assuming there is no interaction between the relative risk factor and the absolute risk like we have shown for body mass index (38). iPrevent currently does this for tamoxifen and risk-reducing surgery, but it can easily be adapted to other risk factors that are not included in the existing risk models. iPrevent uses IBIS v7.02, but IBIS v8.0b, which includes mammographic density, was released in September 2017. Adding mammographic density and SNP data has been shown to make small improvements to the performance of the IBIS model (36). The addition of SNP data also improves BC risk stratification (37). iPrevent uses BOADICEA v3, which does not consider the effects of mutations in genes other than BRCA1 and BRCA2. Potential iPrevent users are warned not to use iPrevent if they have a mutation in another BC predisposition gene. BOADICEA v4 (39) includes the effects of PALB2, CHEK2, and ATM loss-of-function mutations. Pathogenic mutations in these genes are rare, so our study is likely to have included few such women. BOADICEA v5 has recently been published and includes the associations with a 313-SNP polygenic risk score, other lifestyle and hormonal risk factors, and mammographic density (40). Future iterations of iPrevent will consider these recent developments and could also potentially incorporate other new risk factors such as sex hormone levels (41) and double-strand DNA break repair phenotype (42).

A major advantage of iPrevent over other risk estimation tools is that it provides personalized information on BC risk management. Another advantage is that the decision about which BC risk estimation model to use is automated, relieving the user of the need for expert knowledge of the performance characteristics of all the relevant risk estimation models. This enables use of iPrevent by women themselves, or their primary care physicians. Future iterations of iPrevent could include IBIS v8.0b rather than v7.02 because, as we have demonstrated, the former improves calibration, even when mammographic density and polygenic risk are not available. As risk estimation models evolve and improve, iPrevent can be updated to integrate the best performing future models. Although initially designed for use in Australia, future versions of iPrevent could be country specific, aligning risk management information with the guidelines of the country in which iPrevent is being used. Clinical decision support tools like iPrevent may help achieve better precision prevention and screening and ultimately achieve the critical goal of reducing BC incidence and mortality.

Funding

This work was supported by the US National Institutes of Health (NIH), National Cancer Institute (1RO1CA159868); the Breast Cancer Research Foundation; Australia National Health and Medical Research Council (NHMRC) (454508, 288704, 145684); Victorian Health Promotion Foundation; Victorian Breast Cancer Research Consortium; Cancer Australia (809195); National Breast Cancer Foundation (IF 17); Queensland Cancer Fund; Cancer Councils of New South Wales, Victoria, Tasmania, and South Australia; and Cancer Foundation of Western Australia. The six sites of the BCFR were supported by grant UM1 CA164920 from the National Cancer Institute.

KAP is a National Breast Cancer Foundation (Australia) Practitioner Fellow (grant number PRAC-17–004). JLH is an NHMRC senior principal research fellow and a Redmond Barry Professorial Fellow. ACA is funded through Cancer Research UK (C12292/A20861, C12292/A22820). MCS is a NHMRC Senior Research Fellow (1061177).

Notes

Affiliations of authors: Department of Medical Oncology (KAP, SN) and Research Department (kI), Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia; Sir Peter MacCallum Department of Oncology (KAP, kI) and Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health (KAP, RLM, RJM, PCW, AB, GSD, GGG, JLH) and Genetic Epidemiology Laboratory, Department of Clinical Pathology (MCS), University of Melbourne, Parkville, Victoria, Australia; Department of Epidemiology (YL, MBT) and Department of Biostatistics (RB), Mailman School of Public Health, and Departments of Pediatrics and Medicine (WKC) and Herbert Irving Comprehensive Cancer Center (YL), Columbia University Medical Center, New York, NY; Cancer Epidemiology Division, Cancer Council Victoria, Melbourne, Victoria, Australia (RLM, RJM, GGG); Precision Medicine, School of Clinical Sciences, Monash University, Clayton, Victoria, Australia (RLM, MCS); School of Medicine, Deakin University, Geelong, Victoria, Australia (IMC); Lunenfeld-Tanenbaum Research Institute, Sinai Health System, Toronto, ON, Canada (JAK, GG, ILA); Departments of Molecular Genetics and Laboratory Medicine and Pathobiology (ILA) and Dalla Lana School of Public Health (JAK), University of Toronto, Toronto, ON, Canada; Departments of Health Research and Policy and of Biomedical Data Science (ASW) and Department of Medicine and Stanford Cancer Institute (EMJ), Stanford University School of Medicine, Stanford, CA; Department of Dermatology and Huntsman Cancer Institute (DG) and Department of Medicine and Huntsman Cancer Institute (SSB), University of Utah Health, Salt Lake City, UT; Centre for Cancer Prevention, Wolfson Institute of Preventive Medicine, Queen Mary University of London, London, UK (JC); Centre for Cancer Genetic Epidemiology, Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK (ACA); Department of Clinical Genetics, Fox Chase Cancer Center, Philadelphia, PA (MBD).

Conflicts of interest: GSD reports grants from Genetic Technologies Pty Ltd outside the submitted work. KAP and IMC have a patent System and Process of Cancer Risk Estimation (Australian Innovation Patent) issued for the switching algorithm of iPrevent. The data for this study were analyzed independently by YL, RJM, and MBT, not by KAP and IMC. The IBIS model is offered for commercial use by Cancer Research UK, and JC receives a portion of the derived royalties. The other authors declared no conflicts of interest during the conduct of this study outside the grant funding listed in the Funding section. None of the authors were employed by the National Institutes of Health. The content of this manuscript does not necessarily reflect the views or policies of the National Cancer Institute or any of the collaborating centers in the BCFR, nor does mention of trade names, commercial products, or organizations imply endorsement by the US government or the BCFR.

The funding sources had no role in the study. The corresponding author had full access to all the data and the final responsibility to submit for publication.

Author contributions: Conception and design: KAP, YL, MBT. Acquisition of data: KAP, RLM, PCW, SN, WKC, MCS, JAK, ASW, GSD, DG, GGG, GG, ILA, EMJ, MBD, SSB, JLH, MBT. Data analysis: YL, RJM, MBT. Interpretation of data: All authors. Drafting and revising manuscript: All authors. Final approval of manuscript: All authors.

Ethical approval and consent to participate: The cohort protocols were approved by the institutional review boards at each of the respective institutions from which participants were recruited and were conducted in accordance with the Declaration of Helsinki of 2008.