Predicting upstaging of DCIS to invasive disease: radiologist’s predictive performance

Abstract

Objectives:

The purpose of this study is to quantify breast radiologists’ performance at predicting occult invasive disease when DCIS presents as calcifications on mammography and to identify imaging and histopathological features that are associated with radiologists’ performance.

Materials and Methods:

Mammographically detected calcifications that were initially diagnosed as DCIS on core biopsy and underwent definitive surgical excision between 2010 and 2015 were identified. Thirty cases of suspicious calcifications upstaged to invasive ductal carcinoma and 120 cases of DCIS confirmed at the time of definitive surgery were randomly selected. Nuclear grade, estrogen and progesterone receptor status, patient age, calcification long axis length, and breast density were collected. Ten breast radiologists who were blinded to all clinical and pathology data independently reviewed all cases and estimated the likelihood that the DCIS would be upstaged to invasive disease at surgical excision. Subgroup analysis was performed based on nuclear grade, long axis length, breast density and after exclusion of microinvasive disease.

Results:

Reader performance to predict upstaging ranged from an AUC of 0.541 to 0.684 with a mean AUC of 0.620 (95% CI: 0.489–0.751). Performances improved for lesions smaller than 2 cm (AUC: 0.676 vs 0.500; p = 0.002). The exclusion of micro-invasive cases also improved performance (AUC: 0.651 vs 0.620; p = 0.005). There was no difference in performance based on breast density (p=0.850) or nuclear grade (p=0.270)

Conclusion:

Radiologists were able to predict invasive disease better than chance, particularly for smaller DCIS lesions (<2 cm) and after the exclusion of microinvasive disease.

Introduction

Ductal carcinoma in situ (DCIS) is a preinvasive form of breast carcinoma (1, 2). DCIS typically presents as asymptomatic calcifications on mammography and is primarily diagnosed by core needle biopsy (3). The current standard of care is to remove DCIS with definitive surgery (3), but the final surgical pathology may reveal occult invasive disease. The reported rate of upstaging of DCIS to invasive disease after surgical excision is 17–26% (4, 5). The clinical implications of indentifying invasive carcinoma at the time of definitive surgery include the need for additional surgery with sentinel node biopsy and possible chemotherapy (6, 7). In addition, new management strategies for DCIS, such as active surveillance, depend on differentiation of DCIS and invasive disease as surgical excision is not performed (8–12). Preoperative methods that correctly differentiate DCIS from invasive disease are thus needed to provide more personalized care.

Currently the primary means of risk-stratifying women for the presence of occult invasive disease relies on histopathological features from core needle biopsy and some limited radiology features. Histological features such as nuclear grade have been shown to be prognostically relevant (13), with high nuclear grade being associated with higher upstaging of disease (4, 14–22). Although hormone receptors such as estrogen receptor (ER) and progesterone receptor (PR) status have been associated with an increased risk of disease recurrence, their utility to predict upstaging has shown mixed results (4, 23). Radiologists’ assessement of imaging features, such as lesion size and Breast Imaging Reporting and Data System (BI-RADS) classification have been evaluated, but are currently underutilized (24). The combined use of imaging, clinical, and histopathologic features has not been thoroughly tested.

The purpose of this study is to quantify breast radiologists’ performance at predicting occult invasive disease when DCIS presents as calcifications on mammography and to identify imaging and histopathological features that are associated with radiologists’ performance.

Methods

Case Selection

Following Institutional Review Board approval, we queried our institutional database to identify all cases of asymptomatic mammographically detected calcifications that were initially diagnosed as DCIS via a stereotactic core needle biopsy and that subsequently underwent definitive surgical excision between 2010 and 2015. This query did not include women with a history of breast cancer or those with imaging reports describing an associated mass, asymmetry or architectural distortion as these patients have a higher likelihood of invasive disease. This yielded 329 cases of pure DCIS and 68 cases of DCIS upstaged to IDC at surgical excision for an overall upstaging rate of 17.1%. From this master pool, cases were randomly shuffled and the first 120 cases of pure DCIS as well as the first 30 cases of upstaged IDC were selected to maintain a rate of upstaging (20%) typical in clinical practice (4, 5).

Data Collection

From the pre-biopsy diagnostic mammogram report, the BI-RADS breast density was recorded and categorized as low (predominately fatty or scattered fibroglandular) or high (heterogeneously dense or extremely dense) (25). For 13 cases, the breast density was not available as the clinical reports were not available. In these 13 cases, a radiologist who was blinded to the pathology outcome recorded the breast density. From the core needle biopsy pathology report, the nuclear grade, ER, and PR status were recorded. HER2 status is not typically tested for DCIS at our institution. Surgical reports were reviewed to determine the final pathology: DCIS (pure) or invasive cancer (upstaged). The long-axis length of the calcifications was calculated by a computer vision algorithm following annotation by a radiologist.

Radiologist Review

Cases were reviewed in a random order on a Mammography Quality Standards Act certified 5 megapixel Picture Archiving and Communication System monitor (Figure 1). Reader experience ranged from breast imaging fellow-in-training (n = 3) to attending (n = 7) with 3 to 25 years of post-breast imaging fellowship experience. The radiologists were told that all cases were DCIS at core needle biopsy but were blinded to all clinical and pathology information as well as prior images and radiology reports. The diagnostic mammograms that preceded the biopsy, including the magnification views, were available for review. It is standard of practice for all cases of calcifications to have one or more magnification views in conjunction with full field mammogram views available for review. No prior comparison studies were utilized. Readers reported a percentage likelihood of upstaging to invasive disease from 0–100% using a slider scale. Readers also recorded the morphology and distribution of the calcifications according to the BI-RADS Atlas, and reported the presence of a secondary mass, asymmetry, or architectural distortion at the site of the calcifications (25). Of note, no mass, asymmetry, or architectural distortion was reported on the original clinical interpretation, but readers were given the opportunity to report whether they felt one was present in addition to the biopsied calcifications during this review. The option to report secondary features was provided because there is interobserver variability in these features and they are associated with higher upstaging rates.

Figure 1:

(a) 57-year-old woman with pure DCIS. (b) 43-year-old woman with DCIS upstaged to invasive disease at surgical excision.

Statistical Analysis

Fleiss’ coefficient (κ) was used to evaluate the agreement between readers for the BI-RADS morphology and distribution. We utilized area under the receiver operating characteristic curve (AUC) using the pROC_1.0–7 package in R to determine the readers’ ability to predict occult invasive disease upon surgical removal (26). The pROC package utilized DeLong’s method to determine confidence intervals (27). In our analysis, the cancer estimation ratio was considered a prediction probability, thus no training or cross-validation was included. Subgroup analysis was performed based on long-axis length (≤2 cm or > 2 cm), breast density (low or high), and nuclear grade (low/intermediate or high). The 2 cm threshold was used as this differentiates a T1 from a T2 cancer according to the TNM staging system (28). The nuclear grade split into low/intermediate or high was based on the selection criteria used for the active surveillance trials in progress (8–12). Secondary analysis was performed after the exclusion of upstaged cases with microinvasion (defined as no invasive focus >1 mm) as the only form of invasive disease.

Results

The distribution of clinical and pathology variables is shown in Table 1. The average patient age was 58.8 +/− 11.3 years. DCIS was most commonly high nuclear grade (60%), estrogen receptor positive (82%), and progesterone receptor positive (77%). The mean long axis length of DCIS was 16.2 +/− 5.9 mm. There were no significant differences between pure and upstage groups in age, long axis length, breast density, and estrogen receptor/progesterone receptor status. Microinvasion was present on final pathology in 6 cases (4%).

Table 1:

Distribution of clinical, imaging, and histopathological information overall and for cases of pure DCIS and DCIS upstaged to invasive disease at surgical excision.

	Total (n=150)	Pure (n=120)	Upstage (n=30)	p-value
Age (years)	58.8 +/− 11.3	59.3 +/− 11.4	56.9 +/− 10.6	0.243
Long axis length				0.100
Average (mm)	16.2 +/− 5.9	15.9 +/− 6.1	17.2 +/− 4.6
≥2 cm	112 (75%)	92 (77%)	20 (67%)
>2 cm	38 (25%)	28 (23%)	10 (33%)
Density				0.364
Low	61 (41%)	51 (43%)	10 (33%)
High	89 (59%)	69 (58%)	20 (66%)
Nuclear Grade				0.198
Low & Intermediate	60 (40%)	51 (43%)	9 (30%)
High	90 (60%)	69 (57%)	21 (70%)
ER Positive	123 (82%)	101 (84%)	22 (73%)	0.170
PR Positive	116 (77%)	96 (80%)	20 (67%)	0.240
Micro-invasive	6 (4%)	0 (0%)	6 (20%)	N/A

Note – Number refer to counts and column percentage in parentheses or mean +/− standard deviation

Calcifications were most frequently described as fine pleomorphic (53%) and grouped (65%) as shown in Table 2. There was fair agreement for the use of BI-RADS morphology (κ=0.271) and distribution (κ=0.371) descriptors. In addition to the biopsied calcifications, readers reported a secondary mass in 4% of cases, asymmetry in 9% of cases, and architectural distortion in 4% of cases. Of note, the initial clinical reports did not describe masses, asymmetries, or architectural distortion as these were exclusion criteria. Readers reported higher upstaging estimates if they felt there was an associated mass (41% vs 27%), asymmetry (40% vs 27%), or architectural distortion (51% vs 27%). The actual upstaging rates were not significantly higher if a mass (28% vs 20%, p=0.113) or asymmetry (26% vs 19%, p=0.076) was reported, but significantly higher if architectural distortion (37% vs 19%, p<0.001) was reported as shown in Table 2.

Table 2.

BI-RADS descriptor counts with mean reader upstaging estimates and actual upstaging percentages.

Morphology	Count	Upstaging estimate	Actual upstaging
Amorphous	291 (20%)	17%	43 (15%)
Coarse Heterogenous	96 (6%)	29%	18 (19%)
Fine Pleomorphic	794 (53%)	29%	168 (21%)
Fine Linear or fine-linear branching	301 (20%)	34%	70 (23%)
Typical Benign*	10 (1%)	22%	1 (10%)
Distribution
Diffuse	1 (0%)	5%	0 (0%)
Regional	122 (8%)	36%	31 (25%)
Grouped	955 (65%)	25%	185 (19%)
Linear	145 (10%)	27%	26 (18%)
Segmental	255 (17%)	38%	54 (21%)
Mass
Yes	65 (4%)	41%	18 (28%)
No	1435 (96%)	27%	282 (20%)
Asymmetry
Yes	140 (9%)	40%	36 (26%)
No	1360 (91%)	27%	264 (19%)
Architectural distortion
Yes	67 (4%)	51%	25 (37%)
No	1433 (96%)	27%	275 (19%)

Note – The count column refers to the number of times a descriptor was used by any reader for any case.

^*Typically benign BI-RADS morphologies include skin, vascular, coarse, large rod-like, round, rim, dystrophic, milk of calcium, and suture.

Reader performance to predict upstaging of DCIS to invasive disease ranged from an AUC of 0.541 to 0.684 with a mean AUC of 0.620 (95% CI: 0.489–0.751) as shown in Figure 2 and Table 3. Readers performed significantly better (p=0.002) when evaluating lesions 2 cm or smaller (AUC=0.676, 95% CI: 0.517–0.835) compared to lesions larger than 2 cm (AUC=0.500, 95% CI: 0.228–0.772). There was no difference in performance (p=0.104) between low breast density (AUC=0.473, 95% CI: 0.272–0.672) and high breast density cases (AUC=0.681, 95% CI: 0.517–0.846). The performance of readers for low/intermediate nuclear grade DCIS (AUC=0.617, 95% CI: 0.385–0.848) was not significantly different (p=0.27) than high nuclear grade DCIS (AUC=0.614, 95% CI: 0.449–0.778). When cases with microinvasion only were excluded from the upstaged group, the performance of readers significantly improved (AUC=0.651, 95% CI: 0.508–0.795, p = 0.005). There was a non-significant difference (p=0.164) in performance between breast imaging fellows (AUC=0.630, 95% CI: 0.438–0.718) and faculty (AUC=0.630, 95% CI: 0.587–0.673).

Figure 2:

The receiver operating characteristic (ROC) curve for 10 readers is displayed for all 150 cases (n=120 Pure DCIS; n= 30 Invasive). No sub-group analysis is included in this figure.

Table 3:

Reader performance to predict upstaging of DCIS to invasive disease with subgroup analysis.

	N	AUC
Overall	150	0.620 [95% CI: 0.489–0.751]
Lesion Size
<2 cm	38	0.676 [95% CI: 0.517–0.835]
≥2 cm	112	0.500 [95% CI: 0.228–0.772]
Breast Density
Low	61	0.473 [95% CI: 0.272–0.672]
High	89	0.681 [95% CI: 0.517–0.846]
Nuclear Grade
Low & Intermediate	60	0.617 [95% CI: 0.385–0.848]
High	90	0.614 [95% CI: 0.449–0.778]
Microinvasion
All	Upstaged = 30 Pure = 120	0.620 [95% CI: 0.489–0.751]
Excluded	Upstaged = 24 Pure = 126	0.651 [95% CI: 0.508–0.795]

Discussion

Our study analyzed the performance of breast radiologists to predict upstaging of DCIS diagnosed at core needle biopsy to invasive disease at surgical excision. Reader performance overall was better than chance (AUC>0.5) and was improved for cases less than 2 cm in long axis length and after excluding cases of microinvasion, but not improved based on the nuclear grade or breast density.

Individual reader performance was variable ranging from an AUC at the low end of 0.541 up to a maximum AUC of 0.684 (Figure 2). Although all readers were either fellowship trained or current fellows in breast imaging, this is not the same task as differentiating cancer from not cancer that breast imagers are typically asked to perform. Nonetheless, we hypothesized that the skills utilized in clinical practice may translate to this task and the performance of at least some readers suggests that this is possible. However, the range of performances indicates that readers may be incorporating different factors into their prediction estimates. The use of traditional BI-RADS morphology and distribution descriptors suffered from fair interobserver variability, which was likely exacerbated by over half of cases described with a fine pleomorphic morphology and two thirds of cases described with a grouped distribution descriptor. This demonstrates the limited predictive capacity of BI-RADS features. It is possible that readers are incorporating non-BI-RADS morphology and distribution features into their estimates and further work is needed to identify and categorize such features, if they exist.

There were two features that were predictive of improved reader performance: smaller size and the exclusion of microinvasive disease. The improved performance of readers when evaluating lesions 2 cm or less was an unexpected finding since larger lesions provide more calcifications for evaluation. However, larger areas of calcifications may have more heterogeneity in appearance. We theorize that readers may be assessing the calcifications based on their overall appearance which is likely guided by the most common presentation. Since invasive disease can occur in just one small area it may be most appropriate to perform upstaging estimates based on the most suspicious appearance of any one area, no matter how small or unrepresentative of the entirety. Furthermore, biopsy planning might similarly need to be guided to target the most suspicious subgroups of calcifications. Additionally, we found that the exclusion of microinvasive disease improved reader performance. Microinvasion is defined as less than 1 mm of invasive disease and is included with T1 disease for staging purposes (28). However, given the extremely small component of invasion, it is likely that any mammographic features associated with invasion may either not manifest or be too small to appreciate by readers (29). This presents a challenge for the prediction of upstaging and one which may represent a fixed limitation of this type of investigation.

Although all cases selected for this study were originally interpreted as pure calcifications, readers were asked to record any secondary features they thought might be present and in discussion with readers they were fairly liberal in their reporting of these secondary features. When readers reported these secondary features, they notably increased their upstaging estimates, although this increase was only statistically significant for architectural distortion. DCIS presenting as a mass has been previously shown to be predictive of upstaging, but this is a slightly different scenario whereby the primary presentation is calcifications and only a secondary mass was appreciated (4). These findings suggest that close scrutiny for an underlying secondary process may be warranted to guide biopsy planning. However, as our study demonstrated, there is not universal agreement on the presence or absence of these secondary processes. Furthermore, the improved ability of tomosynthesis to evaluate for an underlying secondary process may also be helpful and warrants further evaluation.

The performance of the readers in this study (mean AUC = 0.62) are comparable to those of other investigators who have used alternative methods to predict DCIS upstaging to invasive disease. Computer vision and deep learning algorithms have been applied to mammographic calcifications and shown slightly better AUCs of 0.68 and 0.70 respectively (30, 31). Several investigators have applied clinical and pathological factors to predict upstaging, such as the nomogram developed by Jakub et al. which demonstrated a c-statistic of 0.71 (32). The value of other standard breast imaging modalities such as MRI or even more exploratory modalities such as gamma imaging have been explored in a limited fashion (33, 34). The baseline reader performance described in this study is alone likely not sufficient for immediate clinical use, but it provides a meaningful baseline metric for future studies. Predicting upstaging of DCIS to invasive disease will likely always be a challenging task and the best performance may ultimately be achieved using a combination of imaging, clinical, and histopathological factors. Furthermore, although AUC is a useful overall performance metric for analysis, depending on the specific clinical scenario, it may be beneficial to instead focus on maximizing sensitivity at the cost of specificity (or vice versa). Detailed analyses of these performance tradeoffs and how they might influence clinical treatment decision making are promising areas of future research.

Limitations

Our study has several limitations. Predicting occult invasive disease is a novel task for radiologists and training readers may improve performance. However, it is unclear what features radiologists should be identifying to improve their performance and so further work is needed in order to develop improved training strategies. Additionally, readers knew they were participating in a reader study so their performance might be different than in routine clinical practice, as evident in the use of the secondary descriptors. Radiologists also typically have access to additional clinical, pathological, and radiologic information including one or more prior studies which may influence their prediction performance. There is a very small risk of recall bias in this study as the cases were all imaged 4–9 years prior to the reader study. Finally, subgroup analysis, while informative, is limited by the size of the subgroup populations. For example, 20% of invasive cases demonstrated microinvasion only, thus, reducing the limited upstage cases to 24. Of note, this study was designed to assess reader performance to predict upstaging rather than to determine which clinical, pathologic, or demographic features are predictive of upstaging.

Conclusion

Radiologists were able to predict upstaging of DCIS to invasive disease better than chance, which improved for smaller cases of calcifications and after the exclusion of DCIS with microinvasion. Although radiologists are not currently asked to perform this task, if active surveillance as a management strategy becomes more widespread then radiologists will need to develop the skills to better identify occult invasive disease. However, at baseline the performance of radiologists is comparable to that of recently published manuscripts using quantitative image analysis. Further work is needed to determine if radiologists’ performance can be sufficiently improved through additional training and whether specific imaging features can be identified to improve this performance.

Abbreviations:

AUC

area under the receiver operating characteristic curve

DCIS

ductal carcinoma in situ

ER

estrogen receptor

PR

progesterone receptor