Validation of a Nomogram Predicting Non-Sentinel Lymph Node Metastases among Patients with Breast Cancer after Primary Systemic Therapy - a transSENTINA Substudy

Abstract

Background

Prediction of non-sentinel lymph node (SLN) status after primary systemic therapy (PST) may allow tailored axillary staging. The aim of this analysis was to compare established nomograms from i) the primary operative (n = 6) and ii) the neoadjuvant (n = 1) setting with an optimized nomogram to predict non-SLN status in patients after PST.

Methods

181 patients converting from cN1 prior to PST to ycN0 but found to have a histologically positive SLN in the SENTINA trial were analyzed. Established models were applied. An optimized model was compiled using univariate and subsequent multivariable logistic regression (backward selection, likelihood ratio test).

Results

Area-under-the-curve (AUC) values from the primary operative models showed sufficient performance (0.82–0.71). For the neoadjuvant model, the AUC was found to be inferior to prior analyses (0.66) but within published confidence intervals. The SENTINA nomogram comprised the diameter of the largest lymph node (p = 0.006, odds ratio (OR) = 1.19), tumor size prior to PST (p = 0.085, OR = 1.31), and number of all positive SLN (p = 0.083, OR = 2.04). This model was validated using a separate cohort of arm C (n = 168, AUC 0.79, 95% confidence interval 0.74–0.85).

Conclusion

We validated 7 models of prediction of non-SLN among patients showing axillary conversion through PST. Our own ‘SENTINA nomogram’ yielded AUC values comparable to previous nomograms.

Introduction

The optimal timing for sentinel lymph node biopsy (SLNB) in the setting of primary systemic therapy (PST) is still a matter of debate. Recent studies have analyzed the performance of SLNB in patients that converted from a clinically suspicious sentinel lymph node (SLN) status before to a clinically negative SLN status after PST [1,2,3]. Overall, these studies demonstrated higher false-negative rates (SLN negative, axillary nodes positive) compared to the primary operative setting. Importantly, false-negative rates are dependent upon the number of detected and excised SLN. Adequate prediction of non-SLN involvement should allow for an improved tailoring of axillary surgery after PST without an increased risk of false-negative lymph node assessment. For instance, in cases converting from a cN+ status (i.e., suspicious axillary lymph nodes by palpation and/or ultrasound before PST) to a ycN0 status (i.e., unsuspicious axillary lymph nodes by palpation and/or ultrasound after PST) during PST demonstrating a very low probability of SLN involvement, patients could be counseled in favor of SLNB only and against completion axillary surgery.

In the setting of primary surgery, several nomograms have been developed to predict the probability of non-SLN involvement in the case of a positive SLN. A recent comparative validation in a cohort of 561 patients with a positive SLN suggests differences in model performance, particularly among the subgroup of patients with limited axillary disease burden (micrometastasis or isolated tumor cells) [4]. There is, however, no data analyzing the value of these nomograms in the setting of SLNB following PST.

Recently, a single nomogram was introduced by Jeruss et al. [5] designed to predict SLN status following PST. This nomogram comprised lymphovascular invasion, method of detection of SLN metastasis, multicentricity, suspicious axillary lymph nodes prior to PST commencement, and pathologic tumor size after PST, and was validated successfully in an external patient cohort (area under the curve (AUC) 0.78).

The goal of this study was to analyze the predictive value regarding non-SLN after PST of all nomograms developed i) in the setting of primary surgery (n = 6) and ii) in the post-PST setting within 2 distinct study subgroups among patients who initially presented with clinically positive lymph nodes at the time of diagnosis (cN+) and converted to a clinically negative lymph node status after PST (ycN0) but were found to have lymph node metastases upon SLNB after PST (ypN+).

Methods

Data Assembly

The aim of the SENTINA (SENTinel NeoAdjvant) trial was to provide prospective evidence in a larger population for the use of quality-controlled SLNB in the neoadjuvant setting and to analyze the impact of prior surgery and/or systemic treatment on the success rates of SLNB. The SENTINA study was a 4-arm prospective multicenter cohort study designed to provide reliable data on the feasibility and accuracy of a standardized SLNB procedure in different settings prior to and after PST [2]. Patients could participate if they were scheduled for PST. PST had to consist of at least 6 cycles of an anthracycline-based regimen. Participants were allocated to 4 scenarios according to axillary nodal status before and after PST:

• - Patients with clinically node-negative disease underwent SLNB prior to PST

  • No further axillary surgery was performed in the case of a histologically negative SLN (arm A).
  • A second SLNB after PST followed by axillary dissection was performed in the case of a positive SLN (arm B).
• - Patients with initially node-positive disease underwent immediate PST

  • SLNB and axillary dissection were performed in the case of conversion to a clinically negative axillary status (arm C).
  • Axillary dissection (AD) without SLNB was performed in the case of a remaining positive clinical nodal status, i.e., cN+ before and after PST (arm D).

Assignment of Nomograms and Analysis of Nomogram Performance

In the SENTINA study, the required variables are defined as follows:

• Histologic tumor size was measured in cm after definitive surgery.

• Histologic grade was defined as categorical variable with grades I-III prior to PST.

• Number of SLN resected at definitive surgery.

• Lymphovascular invasion (yes vs. no) at definitive surgery.

• Size of largest SLN metastasis was measured in cm after surgery.

• Extracapsular extension of SLN metastasis (yes vs. no).

• Method of pathologic lymph node detection was defined according to the MSKCC nomogram by van Zee et al. [6] as routine diagnostics or serial sections or immunohistochemistry.

• Tumor type was classified as: invasive ductal, invasive lobular, invasive ductal and lobular, or other.

• Estrogen receptor (ER) positivity (yes vs. no) prior to PST.

• Multifocality (yes vs. no) of the tumor (before) after definitive surgery.

• Presence of macrometastasis (>2 mm; yes vs. no) in the SLN.

• Ratio of positive SLN versus all resected SLN.

The variables were employed to calculate the nomograms. At first, nomograms for the prediction of the probability of positive non-SLN in the context of primary surgery were assessed, i.e., those nomograms designed for patients without prior PST were applied to patients after PST. The following nomograms were analyzed (covariables):

• Cambridge (histologic grade, size of SLN metastasis, ratio of positive SLN to all SLN) [7.]

• Stanford (histologic tumor size, lymphovascular invasion, size of SLN metastasis) [8.]

• Mayo (age, histologic tumor size, number of negative SLN, number of positive SLN, extracapsular extension, size of SLN metastasis) [9.]

• Tenon (histologic tumor size, size of SLN metastasis, ratio of positive SLN to all SLN) [10.]

• MDA (histologic tumor size, lympovascular invasion, number of SLN removed, size of SLN metastasis).

• MSKCC (tumor type, lymphovascular invasion, multifocality, ER status, number of negative SLN, number of positive SLN, method of detection of metastases) [11.]

In addition, the Jeruss nomogram predicting the probability of positive non-SLN in the context of primary systemic therapy was assessed [5]. This nomogram consists of the following parameters: lymphovascular invasion (yes vs. no), method of detection of SLN metastasis (immunohistochemical analysis, hematoxylin and eosin staining, or touch preparation cytology), multicentricity (yes vs. no), clinically positive axillary lymph nodes at diagnosis, and pathologic tumor size.

To compare the predictive performance of the nomograms in our cohort, we selected only patients with full information on all covariables necessary to apply any of the selected nomograms. Nomograms where then applied to patients in arm C to predict the probability of non-SLN involvement. The analysis of the area under the receiver operating characteristic curves (AUC-ROC) was facilitated to assess the nomograms' predictive value. In general, nomograms with larger values (near 1) show better performance (better sensitivity and specificity) compared to nomograms with lower AUC values. AUC values around 0.5 indicate that the nomogram's performance is comparable to a random decision.

Statistical Methods

The patient flow is outlined in a CONSORT diagram. Patient characteristics are described by absolute and relative frequencies for categorical variables. Continuous variables are described by mean and standard deviation. Differences between non-SLN-positive and -negative patients were evaluated by chi-squared tests for categorical variables and Mann-Whitney U tests for continuous variables. The previously published nomograms were applied to our study cohort, and ROC curves were calculated for each nomogram. The AUC was calculated, and 95% confidence intervals (CI) are given for each nomogram. Subsequently, we used the dataset to design a novel nomogram. Multivariable logistic regression models were used to model the probability of detecting a positive non-SLN. Three distinct approaches for nomogram definition were employed.

‘SENTINA Nomogram’

For the development of a new nomogram based on the SENTINA data, we used the same dataset as for applying the previously published nomograms. First, all potential explanatory and relevant covariables were analyzed in a univariate manner using logistic regression models with respect to the outcome of a positive non-SLN. Cofactors were selected for further inclusion in the nomogram in the case of univariate p values of ≤0.2. A backward likelihood ratio variable selection process was used to fit the final model, i.e., the SENTINA nomogram. Covariables were excluded from the model in a stepwise manner where the likelihood ratio test resulted in p values > 0.1. An optimal threshold for the binary decision (non-SLN involvement yes/no) was determined by maximizing Youden's J (J = sensitivity + specificy − 1).

‘Meta Model’

Taking into account the predicted values of the 6 established nomograms and the SENTINA nomogram, we developed a ‘meta model’ . Variables were again selected for further inclusion in the nomogram in the case of a univariate p value of <0.2.

Results

Patients

Table 1 lists the patient characteristics for i) all patients in study arm C, ii) non-SLN-negative patients, and iii) non-SLN-positive patients. Furthermore, a CONSORT diagram describing the compilation of the final dataset is given (supplementary figure 1). Overall, 75 patients with positive SLN following SNB were available for analysis. 41 of these had at least 1 positive non-SLN. Patients were on average 51 years old (range 27–73 years). No difference in age could be identified between the subgroups of patients with positive and negative non-SLN (p = 0.5831, Mann-Whitney U test).

Table 1.

Patient characteristics of the final cohort (arm C)

Parameter	All patients (arm C)	Non-SLN-negative patients	Non-SLN-positive patients	pa
Sample size, n (%)	75 (100)	34 (45.3)	41 (54.7)
Age, years  Mean (SD)	51.8 (10.8)	51.0 (11.0)	52.5 (10.8)	0.58
Median (Q1-Q3)	51.0 (44.0–61.0)	50.0 (44.0–60.0)	51.0 (45.0–62.0)
Diameter of the largest lymph node, mm  Mean (SD)	7.9 (7.2)	4.65 (3.97)	10.63 (8.07)	<0.01
Median (Q1-Q3)	5.0 (3.0–11.0)	4.0 (2.0–6.0)	8.0 (5.0–12.0)
ER-positive, n (%)	56 (74.7)	20 (58.8)	36 (87.8)	<0.01b
Extracapsular extension, n (%)	17 (22.7)	5 (14.7)	12 (29.3)	0.17b
Grading, n (%)  I	2 (2.7)	0 (0)	2 (4.9)	0.02b
II	47 (62.7)	17 (50)	30 (73.2)
III	26 (34.7)	17 (50)	9 (21.9)
Lymph vascular invasion, n (%)	34 (45.3)	12 (35.3)	22 (53.7)	0.11b
Multifocality, n (%)	30 (40)	11 (32.3)	19 (46.3)	0.22b
Tumor size, cm  Mean (SD)	2.6 (2.5)	1.7 (1.4)	3.3 (2.9)	0.0130
Median (Q1-Q3)	1.5 (1.0–3.4)	1.4 (0.8–2.0)	3.1 (1.4–5.0)
Pathological detection method, n (%)  Routine	3 (4)	0 (0)	3 (7.3)	0.06b
Serial	50 (66.7)	27 (79.4)	23 (56.1)
Immunohistochemistry	22 (29.3)	7 (20.6)	15 (36.6)
Resected SLN, n(%)  1–2	38 (50.7)	16 (47.1)	22 (53.7)	0.63
3–4	26 (34.7)	12 (35.3)	14 (34.1)
>4	11 (14.7)	6 (17.6)	5 (12.2)
Positive resected SLN, n (%)  1–2	66 (88)	33 (97.1)	33 (80.5)	<0.01
3–4	8 (10.7)	1 (2.9)	7 (17.1)
>4	1 (1.3)	0 (0)	1 (2.4)
Relative frequency of positive SNL (positive SNL/resected SNL)			<0.01
Mean (SD)	0.72 (0.32)	0.61 (0.35)	0.81 (0.26)
Median (Q1-Q3)	1.00 (0.40–1.0)	0.50 (0.33–1.0)	1.0 (0.66–1.0)

^aMann-Whitney U tests or chi-squared test if not otherwise specified.

^bFisher's exact test.

SD = Standard deviation; ER = estrogen receptor; SLN = sentinel lymph node.

Performance of the Nomograms for the Primary Operative Setting and the Primary Systemic Setting

The performance of all available nomograms is given as a compiled ROC curve in figure 1. Respective AUC values are reported in table 2. All analyzed nomograms showed a reasonable predictive value (AUC > 0.7, for all nomograms p < 0.001 for null hypothesis AUC = 0.5). Analysis of the single available nomogram from the primary systemic setting in our dataset resulted in an AUC value of 0.709 (95% CI 0.081–0.832, p = 0.002).

Fig. 1.

ROS curves for the comparison of the 7 nomograms.

Table 2.

Performance of nomograms from the setting of primary surgery (learning cohort, n = 75)

Nomograms	AUC	Standard error	pa	95% CI of AUC
MSKCC	0.82	0.05	<0.001	0.72–0.92
MDA	0.71	0.06	0.002	0.60–0.83
Mayo	0.71	0.06	0.002	0.60–0.83
Cambridge	0.72	0.06	0.001	0.61–0.84
Tenon	0.75	0.06	<0.001	0.63–0.86
Stanford	0.71	0.06	0.002	0.59–0.82
SENTINAb	0.83	0.05	<0.001	0.74–0.92
Meta nomogram	0.84	0.05	<0.001	0.74–0.93
Full model	0.85	0.04	<0.001	0.76–0.93

^aTesting each nomogram against an AUC of 0.5 (Null hypothesis).

^bIn addition, the SENTINA nomogram was validated in the validation subset (n = 168) with an AUC of 0.83 (95% CI 0.76–0.89, p < 0.001).

AUC = Area under the curve; CI = confidence interval.

Development of an Optimized Nomogram

The final SENTINA nomogram consists of the diameter of the largest lymph node metastasis (p = 0.006, odds ratio (OR) 1.194, 95% CI 1.052–1.352), tumor size prior to PST (p = 0.085, OR 1.309, 95% CI 0.962–1.780), and the absolute number of all SLN that were excised and found to harbor a lymph node metastasis (p = 0.083, OR 2.044, 95% CI 0.911–4.584) as independent risk factors for positive non-SLN. For all 3 risk factors, a higher value, i.e., larger lymph nodes, larger tumor, or increased number of positive SLN, increases the probability of positive non-SLN. We assessed the models' calibration by analyzing the boxplot of predicted probabilities (fig. 2a) and the calibration plot (fig. 2b). Predicted probabilities show a clear separation between the 2 groups of positive versus negative non-SLN. Calibration was moderate within our learning cohort.

Fig. 2.

Results of the application of the SENTINA nomogram depicted as a boxplot of predicted probabilities and b calibration plot; validation of the SENTINA model among 168 previously excluded patients depicted as c ROC curve and d predicted probabilities. Results of the application of the meta model depicted as e boxplot of predicted probabilities and f calibration plot.

We tested the ROC curves of our developed nomogram against the established nomograms. This showed that the SENTINA nomogram performed significantly better than the Mayo (p = 0.05), Cambridge (p = 0.04), and Stanford (p = 0.02) nomograms. No significant difference in performance could be shown when compared to MSKCC (p = 0.88), Tenon (p = 0.13), and MDA (p = 0.06) (table 3).

Table 3.

ROC curve comparison with respect to the SENTINA nomogram (learning cohort, n = 75)

Comparison	AUC differencea	Standard error	95% CI of AUC difference	p
MSKCC versus SENTINA	-0.007	0.044	-0.093 to 0.079	0.88
Mayo versus SENTINA	-0.114	0.058	-0.229 (to −0.000)	0.05
Cambridge versus SENTINA	-0.107	0.052	-0.208 (to −0.005)	0.04
Stanford versus SENTINA	-0.122	0.051	-0.2226 (to −0.022)	0.02
Tenon versus SENTINA	-0.083	0.055	-0.190 to 0.025	0.13
MDA versus SENTINA	-0.115	0.061	-0.235 to 0.005	0.06

^aNegative values indicate superiority of the SENTINA nomogram. AUC = Area under the curve; CI = confidence interval.

We validated the model using patients excluded from the dataset due to missing values for the covariates used in the other nomograms. In total, 168 of the previously excluded patients could be included in the validation cohort. The AUC for the validation cohort was 0.79 (95% CI 0.74–0.85), reflecting a slightly lower but still good prediction of positive non-SLN (fig. 2c). The calibration plot shows quite a good calibration; however, patients with low predicted probabilities have more often a positive non-SLN (fig. 2d).

A reasonable cut-off value with respect to sensitivity and specificity is 45%.

After variable selection, the final model consisted of a combination of the MSKCC model (OR 1.07, 95% CI 1.02–1.11, p = 0.004) and the Mayo model (OR 1.02, 95% CI 1.00–1.03, p = 0.061). We did not include the new SENTINA model since it was tested on the same dataset, and overfitting was likely to occur. Figures 2e and f show the predicted probabilities and the calibration curve, respectively. Calibration is better than in the SENTINA nomogram.

Discussion

The optimal timing of SLNB in the context of PST still needs to be defined. While SLNB before PST has reliable validity, conducting SLNB post-PST might spare patients a second surgery (i.e., axillary dissection), provided it is sufficiently valid. If patients with a low enough risk of non-SLN metastases could be identified, omission of completion axillary dissection after SLNB could become a reasonable clinical option. The concept of SLNB has been recently analyzed by 2 large clinical trials [1,2]. We here present the results of an analysis based on one of those 2 trials (i.e., the SENTINA study), which was set to develop an optimized algorithm for the evaluation of SLN status among patients with breast cancer undergoing preoperative systemic therapy. In addition to testing several nomograms developed for the primary operative setting and 1 nomogram for the post-neoadjuvant setting, we used our dataset to develop our own optimized nomogram.

Overall, estimated AUC values regarding the 6 nomograms from the primary operative setting are comparable to results obtained in a systematic validation analysis reported previously [3]. However, estimated CIs are slightly larger due to the smaller number of analyzed patients in the transSENTINA study. In comparison, validation of the Jeruss nomogram in the post-neoadjuvant setting yielded AUC values lower than those in the original validation cohort and lower than those of nomograms from patients with primary surgery (MSKCC), suggesting predictive inferiority compared to others [4]. Our results show that nomograms predicting non-SLN status after PST need to be optimized to avoid full axillary dissection in patients after PST, especially in those who clinically convert from cN+ to ycN0 after PST.

In order to enhance the predictive capacity of non-SLN nomograms, we developed 2 novel nomograms and tested them on the present dataset. Both the SENTINA nomogram and the meta model performed reasonably well. It is not surprising that the meta model showed the best predictive performance given that it may have been overfitted due to development and testing based on the same dataset. Therefore, validation using an independent dataset is urgently warranted.

Our analysis has limitations. Most of all, the nomogram analysis was impaired due to the low number of included patients. This was due to the low probability of patients in arm C having additional lymph node metastases, reflecting the application of modern chemotherapy in the SENTINA trial with an increased probability of complete axillary response to preoperative chemotherapy.

In summary, our data suggests that the nomograms examined might be of value in deciding for/against axillary dissection in patients following PST. Before introducing these nomograms, and particularly the SENTINA nomogram developed in this analysis, additional validation of this data in independent datasets is planned.

Online Supplemental Material

Suppl. table 1. Development of the SENTINA nomograms.

Suppl. table 2. Patient characteristics of the validation cohort.

Suppl. table 3. Sensitivity and specificity of different cut-off values for the SENTINA nomogram (learning cohort, n = 75).

Suppl. fig. 1. Consort diagram describing development of the patient population.

Suppl. fig. 2. Development of a reasonable cut-off value for the SENTINA nomogram with respect to sensitivity and specificity.

To access the supplemental material please refer to www.karger.com/?DOI=489565.

Previous Presentation

This analysis has been presented prior to this publication at the ASCO Annual Meeting 2013 and the Annual Meeting of the German Society of Senology 2014 (Deutsche Gesellschaft für Senologie (DGS)).

Disclosure Statement

The SENTINA study received financial and logistic support from AGO-B, Brustkrebs Deutschland, and the German Breast Group (GBG).

Supplementary Material

Supplementary data