External validation of IBTR! 2.0 nomogram for prediction of ipsilateral breast tumor recurrence

Abstract

Purpose

IBTR! 2.0 nomogram is web-based nomogram that predicts ipsilateral breast tumor recurrence (IBTR). We aimed to validate the IBTR! 2.0 using an external data set.

Materials and Methods

The cohort consisted of 2,206 patients, who received breast conserving surgery and radiation therapy from 1992 to 2012 at our institution, where wide surgical excision is been routinely performed. Discrimination and calibration were used for assessing model performance. Patients with predicted 10-year IBTR risk based on an IBTR! 2.0 nomogram score of <3%, 3%–5%, 5%–10%, and >10% were assigned to groups 1, 2, 3, and 4, respectively. We also plotted calibration values to observe the actual IBTR rate against the nomogram-derived 10-year IBTR probabilities.

Results

The median follow-up period was 73 months (range, 6 to 277 months). The area under the receiver operating characteristic curve was 0.607, showing poor accordance between the estimated and observed recurrence rate. Calibration plot confirmed that the IBTR! 2.0 nomogram predicted the 10-year IBTR risk higher than the observed IBTR rates in all groups. High discrepancies between nomogram IBTR predictions and observed IBTR rates were observed in overall risk groups. Compared with the original development dataset, our patients had fewer high grade tumors, less margin positivity, and less lymphovascular invasion, and more use of modern systemic therapies.

conclusions

IBTR! 2.0 nomogram seems to have the moderate discriminative ability with a tendency to over-estimating risk rate. Continued efforts are needed to ensure external applicability of published nomograms by validating the program using an external patient population.

Introduction

As the incidence of breast cancer has increased constantly, the proportion of breast-conserving surgery (BCS) has increased rapidly since 2002 in Korea [1,2]. Following thyroid cancer, breast cancer is the second most common female cancer in Korea. Radiotherapy (RT) has been recommended as a standard treatment for several decades and proved to reduce 10-year risk of any first recurrence and 15-year breast cancer death risk [3,4]. Considering the importance of quality of life and cosmetics, BCS and RT is preferred over a mastectomy approach.

Nomograms are a predictive tool that can provide an overall expectation of a specific outcome. Nomograms are widely used to assess numerous malignancies and to estimate cancer prognosis; the IBTR! is a popular nomogram that can predict ipsilateral breast tumor recurrence (IBTR) after BCS. The IBTR! 1.0 nomogram is the initial version of the nomogram that uses seven prognostic factors: age, tumor size, tumor grade, margin status, lymphovascular invasion (LVI), chemotherapy usage and hormone therapy usage [5]. The IBTR! 2.0 was optimized with data on 7,811 patients found in the British Columbia Cancer Agency (BCCA) breast cancer database between 1989 and 1999 [6]. Despite its clinical utility, there is a potential risk that nomograms overestimate and underestimate the actual rate. Before implementing the nomogram as useful clinical tool at any institution, objective validation with a specific dataset is critically important.

In our institution, wide surgical excision has been routinely conducted. Breast surgeons resect enough margins encompassing the entire cavity of the tumor bed, providing less local recurrence than currently recommended boundaries for inked-margins [7]. This study aimed to evaluate the efficacy of the IBTR! 2.0 nomogram at our institution.

Materials and Methods

1. Patients

From January 1991 to December 2012, a total of 2,206 patients who received BCS with RT at Yonsei Cancer Center were reviewed retrospectively. Patients with in situ tumors, with distant metastasis at the time of diagnosis, or patients who received BCS without RT were excluded. Approval for this study was granted by the Institutional Review Board of Yonsei University Hospital (No. 2017-1146-001).

2. Treatment

BCS was performed by wide excision of the primary tumor with a safety margin over 1 cm. The resection margin was shaved all around to eliminate any remnant breast tissue. The resection margin was marked with blue-colored ink far from the originally located tumor site and evaluated by the frozen section analysis. Microscopically involved margins were reexcised to achieve the negative margin. Along with breast surgery, sentinel lymph node biopsy or standard axillary lymph node dissections were completed. From early 2000, sentinel lymph node biopsy using radioisotopes have been performed. Patients with positive sentinel lymph nodes underwent axillary node dissection.

Patients received whole breast RT using a tangential technique after BCS. Target delineation was performed using the European Society for Radiotherapy and Oncology guidelines [8,9]. A total of 50.4 Gy with 1.8–2 Gy per fraction was applied for the whole breast with or without regional nodal area. Four to 6 MV X-ray linear accelerators were used for whole breast irradiation. Sequentially, 10 Gy with 2 Gy per fraction for a boost to the tumor bed were performed with an electron beam. For patients with positive axillary lymph node or medially located tumors, regional node irradiation was performed including supraclavicular lymph nodes, internal mammary lymph nodes and axillary lymph nodes [10].

Selective estrogen receptor modulators or aromatase inhibitors were recommended in patients with hormone receptor positive tumors. Trastuzumab has been recommended for use in human epidermal growth factor receptor 2 positive tumors since 2005 at our institution. The chemotherapy regimen was determined according to standard treatment guidelines. Patients with a tumor size larger than 0.5 cm or positive regional lymph nodes received Adriamycin and cyclophosphamide every 21 days for 4 cycles with an additional 12 cycles of paclitaxel every week. Omitting the paclitaxel was considered in women with negative regional lymph nodes.

3. Follow-up

Physical examination, radiologic imaging studies, laboratory tests were performed for follow-up every 6 months for 5 years, and then every year up to 10 years. Ultrasound imaging was performed every 6 months, and a mammography was conducted every year to evaluate locoregional recurrence up to 5 years. Magnetic resonance imaging was performed at 2 and 5 years after BCS.

4. Study endpoint and statistical analysis

The primary endpoint of this study was to validate the IBTR! 2.0 nomogram. In this study, the IBTR was defined as tumor recurrence in the ipsilateral breast. The secondary endpoint was to determine the patterns of recurrence in patients receiving BCS and RT.

All estimated 10-year IBTR in this study were calculated using the IBTR! 2.0 website application. To evaluate model performance, we assessed discrimination and calibration. Discrimination refers the ability to discriminate classes of outcomes. To evaluate discriminative ability of this model, receiver operating characteristic (ROC) and area under the curve were utilized.

Calibration helps to demonstrate how far the estimation value is from the actual outcomes. In this study, calibration plots were used to assess the calibration by the observed IBTR rate against the nomogram-derived IBTR probabilities; observed and predicted outcomes form a 45° line in a perfectly calibrated nomogram. Also, we showed the Kaplan-Meier IBTR curves of 4 risk groups according to their estimated 10-yr IBTR rate by nomogram: group 1, <3%, group 2, 3%–5%; group 3, 5%–10%; group 4, >10%.

Overall survival (OS) and disease-free survival (DFS) rates were also calculated using Kaplan-Meier methods. The OS was defined as the time interval between the date of breast surgery and the date of death. The DFS was defined as the time from the surgery to the first locoregional recurrence, systemic recurrence or death. Univariate and multivariate analysis using the Cox proportional hazard model was performed to evaluate the factors affecting the OS, DFS, and IBTR independently. A p-value less than 0.05 were judged as statistically significant. Statistical analysis was performed using SPSS software version 23.0 (IBM, Armonk, NY, USA), SAS version 9.4 (SAS Institute, Cary, NC, USA), and R package version 3.1.3.

Results

1. Patients and tumor characteristics

Patients and tumor characteristics are presented in Table 1. The median age of the patients was 49 years (interquartile range, 43 to 56 years). Compared with the BCCA cohort involved in creating the IBTR! 2.0 nomogram, patients had less positive resection margin (10.5% vs. 2.5%), less LVI (26.9% vs. 5.60%) in our study cohort, respectively. More patients received chemotherapy (29.1% vs. 64.2%) and hormone therapy (39.4% vs. 73.4%), respectively. We also compared patients and tumor characteristics with patients at the University Hospital Leuven. Concordant with the BCCA cohort, less positive resection margins (5.0% vs. 2.5%) and less LVI (17.9% vs. 5.6%) were observed. In addition, more chemotherapy (29.6% vs. 64.2%) was performed in our study cohort when compared with the University Hospital Leuven cohort (Table 1).

Table 1.

Comparison of patient characteristics with BCCA cohort and University Hospital Leuven cohort (development set of IBTR! 2.0 nomogram)

	YCC (n = 2,206)	BCCA [6] (n = 7,811)	University Hospital Leuven cohort [11] (n = 1,898)
Age (yr)	49 (20–82)	58 (17–88)	57 (22–90)
Tumor size (cm)
≤1	591 (26.80)	2,096 (26.80)	398 (20.97)
1.1–2	1,086 (49.20)	3,675 (47.00)	835 (43.99)
>2	529 (24.00)	2,040 (26.10)	665 (35.04)
Nodal statusa)
Negative	1,753 (79.80)	-	1,342 (72.00)
Positive	445 (20.20)	-	521 (28.00)
Gradea)
1	564 (27.90)	1,359 (18.10)	361 (19.03)
2	936 (46.30)	3,593 (47.90)	835 (44.02)
3	520 (25.70)	2,544 (33.90)	701 (36.95)
Resection margina)
Negative	2,135 (96.90)	6,730 (89.30)	1,508 (79.70)
Close (≤ 2 mm)	12 (0.50)	15 (0.20)	290 (15.30)
Positive	56 (2.50)	788 (10.50)	94 (5.00)
Lymphovascular invasiona)
No	2,082 (94.40)	5,516 (73.10)	1,002 (82.10)
Yes	123 (5.60)	2,025 (26.90)	218 (17.90)
Chemotherapy
No	790 (35.80)	5,536 (70.90)	1,337 (70.40)
Yes	1,416 (64.20)	2,275 (29.10)	561 (29.60)
Hormonal therapy
No	587 (26.60)	4,731 (60.60)	351 (18.50)
Yes	1,619 (73.40)	3,080 (39.40)	1,547 (81.50)
Tumor laterality
Right	1,074 (48.70)	-	-
Left	1,132 (51.30)	-	-
Tumor locationa)
Outer and central	1,447 (66.50)	-	-
Inner	730 (33.50)	-	-

Values are presented as median (range) or number (%).

YCC, Yonsei Cancer Center; BCCA, British Columbia Cancer Agency.

^a)Missing data were excluded.

2. Recurrence

Table 2 provides the results concerning recurrence. The median follow-up period was 6.09 years (range, 6 to 23.2 years). Among 2,206 patients, 116 patients experienced recurrence. Distant metastasis was a dominant pattern of failure in patients who received BCS and RT. IBTR was occurred in 16 patients. We defined the IBTR as local recurrence without regional or distant metastasis. Among the local recurrence group, there were 3 true recurrences and 8 that occurred elsewhere. Regional failure was observed in 15 patients. Approximately half of the patients in regional recurrence group experienced supraclavicular node recurrence. Among the patients who experienced distant metastasis, 15% of patients underwent simultaneous locoregional recurrence. The 10-year OS, DFS, and cancer specific survival rates were 94.6%, 93.7%, and 96.9%, respectively, as shown in Fig. 1.

Table 2.

Patterns of treatment failure after breast conserving therapy (n = 116)

Recurrence	No. of patients
Locoregional recurrence	31
Local recurrence	16
Near primary site	5
Elsewhere	8
Skin	3
Regional recurrence	15
SCL	7
IMN	1
AXL	6
SCL + IMN	1
Distant metastasis	72
Locoregional + distant metastasis	13

SCL, supraclavicular lymph node; IMN, internal mammary; AXL, axillary lymph node.

Fig. 1.

Kaplan-Meier plot of overall survival (OS), cancer-specific survival (CSS), and disease-free survival (DFS).

3. Factors related to IBTR, OS, and DFS

We conducted a Cox univariate analysis to evaluate the factors affecting IBTR, OS, and DFS. Due to the limited number of IBTR events in our study, hormone therapy was the only factor affecting IBTR (hazard ratio [HR] = 0.22; 95% confidence interval [CI], 0.08–0.61; p = 0.004) (Table 3).

Table 3.

Prognostic factors affecting ipsilateral breast tumor recurrence rate

Variable	Univariate analysis
	No. of event (n = 16)	HR (95% CI)	p-value
Age (yr)		1.00 (0.95–1.05)	0.906
pT stage		1.82 (0.66–5.02)	0.246
T3–4	10 / 532
T1–2	6 / 1,670
pN stage		0.23 (0.31–1.76)	0.159
N+	1 / 445
N0	15 / 1,753
Location		0.86 (0.29–2.48)	0.781
Inner	5 / 730
Outer and central	11 / 1,447
Tumor grade		1.57 (0.77–3.20)	0.208
3	6 / 520
2	6 / 936
1	3 / 564
Resection margin		1.56 (0.56–4.37)	0.395
Negative	15 / 2,135
Close (≤2 mm)	0 / 12
Positive	1 / 56
Lymphovascular invasion		1.20 (0.16–9.11)	0.858
Positive	1 / 123
Negative	15 / 2,082
Chemotherapy		0.87 (0.31–2.41)	0.796
Yes	10 / 1,416
No	6 / 790
Hormone therapy		0.22 (0.08–0.61)	0.004
Yes	6 / 1,619
No	10 / 587

HR, hazard ratio; CI, confidence interval.

For the univariate analysis, tumor stage (T stage), node stage (N stage), tumor grade, LVI were independent prognostic factors for OS (Supplementary Table S1). Based on factors identified as significant by the univariate analysis, we performed the multivariate analysis, which showed that node positive status (HR = 1.69, 95% CI, 1.03–2.75; p = 0.037) was a prognostic factor of OS. Grade 3 tumors were a poor prognostic factor compared with grade 1 tumors (HR = 2.61; 95% CI, 1.26–5.40; p = 0.01) (Supplementary Table S2).

Additional analysis was performed to assess the independent prognostic factors for DFS. The T stage, N stage, and the tumor grade were significant factors affecting DFS on univariate analysis (Supplementary Table S1). For the multivariate analysis, the N stage (HR = 1.70; 95% CI, 1.18–2.24; p = 0.004) was the only independent prognostic factor for DFS (Supplementary Table S2).

4. Validation

To validate the web-based IBTR! 2.0 nomogram, we assess the data with respect to discrimination and calibration. To evaluate discriminative ability of this nomogram, we conducted c-statistics. The area under ROC curve was 0.607 in this nomogram indicating it has poor discrimination.

Calibration plots of the IBTR! 2.0 nomogram is shown in Fig. 2. Each figure represents the relationship between the nomogram estimated IBTR rate and the observed IBTR rate. The 45° line represents the ideal nomogram, demonstrating that the nomogram predicted outcome would be concordant with the observed outcome. The error bar in calibration plot represents the IBTR! 2.0 nomogram performance when applying data from our study database. A Kaplan-Meier analysis for IBTR rate was done based on four risk group. The graph for Kaplan-Meier analysis is shown in Fig. 3. Four groups were classified according to predicted 10-year IBTR risk by the IBTR! 2.0 nomogram. Large discrepancies between nomogram IBTR predictions and observed IBTR rates were observed in overall risk groups. The expected and observed IBTR rate was 2.1% and 0.6% in group 1, 3.8% and 0.9% in group 2, 6.5% and 1% in group 3, and 11.2% and 0% in group 4, respectively.

Fig. 2.

Calibration plot for IBTR! 2.0 nomogram. The nomogram predicted recurrence rate was plotted against the observed recurrence rate.

Fig. 3.

Kaplan-Meier plot for observed ipsilateral breast tumor recurrence (IBTR) in four risk groups, based on predicted recurrence less than 3%, 3%–5%, 5%–10%, and higher than 10%.

Discussion and Conclusion

The IBTR! 2.0 nomogram is a predictive tool for local recurrence after breast conserving therapy. There are several researchers validating the IBTR! 2.0 nomogram using their own database [6,11,12]. Nevertheless, the clinical utility of IBTR! 2.0 nomogram should continue to be tested under heterogeneous patient population conditions and clinical settings [13]. Considering discordant findings from mentioned validation studies, we believe our findings validating the IBTR! 2.0 nomogram with our own dataset add meaningful knowledge to the existing literature

In the current study, the IBTR! 2.0 nomogram showed poor discriminative abilities with a C-statics value of 0.607. This value is consistent with study from University Hospital Leuven that had C-static values between 0.57 and 0.59 [11]. However, this result was slightly lower compared with a study from BCCA and Samsung Medical Center (C-statics = 0.66 and 0.67 respectively). In studies from BCCA and the Samsung Medical Center, approximately 5% of patients experienced IBTR [6,12]. However, Leuven study presented 1.8% of IBTR rate showing much less recurrence rate [11]. Our study showed only 0.7% of local recurrence, demonstrating a huge discrepancy with the original article and the Samsung Medical Center study. Taken together, IBTR! 2.0 nomogram seems to have modest discriminative ability for predicting the IBTR risk rate. With respect to modest discriminative abilities in our study, an extremely low rate of events would be main reason.

In cohort studies from Massachusetts General Hospital and the BCCA which created the IBTR! 2.0 nomogram, low and moderate risk patients seemed to accurately predict the IBTR risk rate by the IBTR! 2.0 nomogram. However, in patients with high risk features, the IBTR! 2.0 nomogram overestimated the risk [6]. In the current study, the calibration results showed that IBTR! 2.0 nomogram overestimated the IBTR risk in overall risk groups. Consistent with our results, the University Hospital Leuven results indicated overestimation of IBTR risk in all risk groups [11]. Contrary to BCCA and our study, Samsung Medical Center data demonstrated an underestimation in the higher risk group. However, low and moderate risk groups showed high correlation between actual and estimated IBTR risk [12]. This implies that the calibration results of IBTR! 2.0 nomogram could be varied across the institutions.

The reason for disparity in calibration might originate from several factors. There are some differences in patient and tumor characteristics when compared to other study groups. Our results showed a much lower IBTR rate than other studies. Our study cohort had fewer high grade histology tumors, more use of chemotherapy and more use of hormonal treatment compared with the BCCA cohort. Other differences also existed such as resection margins (positive margin, 2.50% vs. 10.50%), LVI (5.60% vs. 26.90%) and tumor grade (grade 3, 25.70% vs. 33.90%). The LVI rates seem to be much lower in our institution. This may be partly explained by LVI detection which is somewhat subjective [14].

Wide margin resection performed in our institution is done to obtain additional tissue from the circumferential margin of the cavity after lumpectomy. Randomized, controlled trial conducted at the Yale cancer center proved that wide margin resection tended to reduce the odds of positive margins (OR = 2.06; 95% CI, 0.98–4.32; p = 0.06) [7]. Considering that resection margin status has been closely associated local recurrence of breast cancer, wider resection margin in our institution might attribute to the overall low rate of IBTR [15-17]. Another possible reason is the use of modern chemotherapy. With respect to chemotherapy, there was high discrepancy in the HR of chemotherapy use between our cohort and the model cohort. The HR of chemotherapy in our cohort was 0.37 in the multivariate Cox-analysis (data not shown), which seems to be much lower than the original report [6]. Additional paclitaxel for adjuvant chemotherapy offered better local control than adjuvant doxorubicin/cyclophosphamide with comparable toxicity [18]. In the trastuzumab era, trastuzumab is effective for reducing locoregional recurrence with human epidermal growth factor receptor 2 positive tumors [19]. Also, patients who received hormone therapy had less locoregional recurrence than patients without hormone therapy [20,21]. Taken all together, development of systemic therapy such as, chemotherapy, hormone therapy and trastuzumab might contribute to lowering the rate of IBTR, which led to an underestimation.

There are several limitations inherent to the retrospective nature of this study, even though our study registry has been prospectively collected data. As our cohort data was collected for a long period, from 1999 to 2012, there were several changes over time, such as criteria for positivity of hormone receptor or LVI, and changes in systemic treatments overtime. Secondarily, a small number of recurrence event limits the complete statistical analysis. Finally, our median follow-up time was 73 months, relatively short follow-up period in breast cancer. These factors could underestimate the IBTR risk.

In conclusion, this study showed that the IBTR! 2.0 nomogram seems to have a poor discriminative ability with a tendency to over-estimate risk rate. Wide resection margins and use of modern chemotherapy in our cohort might attribute to an overall low IBTR rate, which might be the main reason for the underperformance of the IBTR! 2.0 nomogram. Caution is needed to interpret the predicted value before validating the nomogram by their own dataset.

Supplementary Materials

Supplementary materials can be found via https://doi.org/10.3857/roj.2018.00059

Table S1. Kaplan-Meier estimated 10-year OS and DFS.

Table S2. Multivariable regression analysis of overall survival and disease free survival.