Audit of prior screening mammograms of screen-detected cancers: Implications for the delay in breast cancer detection

Abstract

When cancer is detected in a screening mammogram, on occasion retrospective review of prior screening (pre-index) mammograms indicates a likely presence of cancer. These missed cancers during pre-index screens constitute a delay in detection and diagnosis. This study was undertaken to quantify the missed cancer rate by auditing pre-index screens to improve the quality of mammography screening practice. From a cohort of 135 screen-detected cancers, 120 pre-index screening mammograms could be retrieved and served as the study sample. A consensus read by two radiologists who interpreted the pre-index screens in an unblinded manner with full knowledge of cancer location, cancer type, lesion type, and pathology served as the truth or reference standard. Five radiologists interpreted the pre-index screens in a blinded manner. Established performance metrics such as sensitivity and specificity were quantified for each reader in interpreting these pre-index screens in a blinded manner. All five radiologists detected lesions in 8/120 (6.7%) screens. Excluding the two readers whose performance was close to random, all the 3 remaining readers detected lesions in 13 pre-index screens. This indicates that there is a delay in diagnosis by at least one cycle from 8/120 (6.7%) to 13/120 (10.8%). There were no observable trends in terms of either the cancer type or the lesion type. Auditing prior screening mammograms in screen-detected cancers can help in identifying the proportion of cases that were missed during interpretation and help in quantifying the delay in breast cancer detection.

INTRODUCTION

Missed breast cancers are a common cause of medicolegal lawsuits against breast imaging radiologists (1,2). A missed cancer diagnosis may be construed as a medical error. Medical errors are the third leading cause of death in the United States (3). What constitutes missed breast cancer? When an obvious imaging finding at the location of the current tumor is identified in retrospect on a prior mammogram and should have prompted a diagnostic workup would constitute a missed cancer (4). Not uncommonly following a “negative” screening mammogram, an asymptomatic woman discovers an imaging abnormality on a subsequent screening mammogram, that proves to be cancer. Occasionally following a “negative” mammogram and prior to her next scheduled screening mammogram a woman may be symptomatic, and the ensuing workup reveals a cancer diagnosis. Labels such as “missed cancer”, “interval cancer” and “false negative” mammograms have been used to describe such errors. While these are distinct terms, they are sometimes used interchangeably. This error rate varies from 20 to 58% (4,5,6). Hoff et al., (5) in their study of Norwegian screening records observed that 24% screen-detected cancers had discordance between 2 independent readers (all screens are double-read in Norway).

Mammograms remain an important tool for screening of breast cancers. Screening mammograms have been shown to reduce mortality by up to 40% (7,8). Mammograms however have sensitivities ranging from 71–96% for the detection of breast cancers and this sensitivity drops to 48% in dense breasts (9,10,11). Apart from density, mammographic texture also contributes to sensitivity, with heterogeneous texture reducing the sensitivity (9,10). Breast cancer is a low-prevalence disease in the population. Apart from the low prevalence, the abnormal findings on a screening mammogram are often subtle, making the diagnosis a visually challenging task and therefore error prone (12). The goal of any breast imager is to be able to diagnose cancers at an early stage when they are most amenable to cure with less morbid treatment options (13). Errors are broadly classified into two categories one of perception where the radiologist fails to notice or act on a finding, and the second of image analysis or interpretation where the radiologist observes the finding but considers them stable, non-specific, or benign, and not meriting additional views or action. Other contributing factors related to technique, patient, and reader factors have been described in the literature (6,11,14).

As a quality audit to improve radiologists’ performance in our own practice we decided to retrospectively review the prior screening mammogram of 135 screen-detected breast cancers. Of these 135 screen-detected breast cancers, 120 pre-index screens (between July 2011-June 2013) performed 12–24 months prior to the index study were available. All of these were 2D full-field digital mammograms (FFDM). Our study design described in Methods and Material was different from those described in the literature (4,5,6). Two board-certified, fellowship-trained radiologists reviewed not only the current and all prior images but also reviewed all prior biopsy reports available in the electronic health records to establish the ground truth. Five board-certified, fellowship-trained breast imagers blindly reviewed the bilateral, standard 2-view 2D prior FFDM screening mammograms intermixed with 75 age- and density-matched normal 2D FFDMs. Normal mammograms are defined as those having two subsequent negative screening exams (BI-RADS 1 or 2). In this work, we focus on the prior screening mammograms. The goal of our study was to not only provide learning opportunities from our misses regarding the types of lesions being missed but also study the generalizability of these misses. We also wanted to determine if there are differences in lesion pick-up rate among pre-index screens when reviewed in a blinded manner by individual radiologists’ (single read) in comparison to unblinded consensus-read.

MATERIALS AND METHODS

The study design is illustrated in Figure 1. A total of 135 women with screen-detected breast cancers were included. The mammograms from the previous round of screening referred to as pre-index screens were retrieved from the picture archiving and communication system (PACS). Pre-index screening mammograms could not be retrieved in 15 subjects and hence 120 pre-index screens were available for the study.

Figure 1.

Flowchart of the study design.

Unblinded consensus read:

The purpose of this consensus read is to establish the truth for the presence of a lesion, and the location (laterality, quadrant) of the lesion and the lesion descriptor (mass, calcifications, architectural distortion, and asymmetry) if present, in pre-index screening mammograms. During this consensus read all available information, including the screening mammograms that led to cancer diagnosis, images from diagnostic workups, and pathology reports were used for determination. Two board-certified, breast imaging fellowship-trained radiologists with 2 and 15 years of experience performed the consensus read.

Individual blinded read:

Five board-certified breast imaging fellowship-trained radiologists with 2–12 years of experience served as readers. Only the pre-index screening mammograms were provided to the readers. Each reader reported the presence/absence of a lesion, the location of the lesion, and the lesion descriptor if present.

Data preparation:

If the individual blinded read of a pre-index screen indicated the presence of a lesion, it was considered a true-positive if the location (laterality and quadrant) was concordant with the unblinded consensus read. If the individual blinded read detected a lesion but was discordant with the unblinded consensus read in terms of lesion presence/absence and location, then it was considered a false-positive. If the individual blinded read did not detect a lesion, whereas one was detected in the unblinded consensus read, then it was considered a false-negative. If both the unblinded consensus read and the individual blinded read did not detect a lesion, then it was considered a true-negative. The lesion descriptor was not included in the above determinations due to known variability among readers.

For each reader, the standard performance metrics including sensitivity, specificity, precision, and accuracy for interpreting the pre-index screens were computed. Since the study used a binary decision (lesion presence/absence) for the individual read the operating point of each reader in terms of their sensitivity and specificity was plotted in a receiver operating characteristic (ROC) plot. The proportion of pre-index screens that were correctly detected by all individual readers and the proportion of pre-index screens that were not detected by any individual reader were determined and their associations with cancer types and lesion descriptors were analyzed. Cancer types were classified as invasive ductal carcinoma (IDC), invasive lobular carcinoma (ILC), ductal carcinoma in situ (DCIS), and other types of cancers. Lesion descriptors were broadly classified as soft tissue lesions (mass, architectural distortion, and asymmetry), soft tissue lesions with calcifications, and calcifications only.

RESULTS

The subject characteristics including age and breast density categories for all pre-index screens, pre-index screens with a positive consensus read, and pre-index screens with a negative consensus read are shown in Table 1. The age and breast density categories are similar across the three cohorts.

Table 1.

Distribution of age and density categories.

	Pre-index screens (n=120)	Positive consensus read (n=51)	Negative Consensus read (n=69)
Age (years)	62.3 ± 10.2	64.3 ± 9.8	60.8 ± 10.3
Density categories
- BI-RADS A	8/120 (7%)	4/51 (8%)	4/69 (6%)
- BI-RADS B	65/120 (54%)	27/51 (53%)	38/69 (55%)
- BI-RADS C	42/120 (35%)	18/51 (35%)	24/69 (35%)
- BI-RADS D	5/120 (4%)	2/51 (4%)	3/69 (4%)

The standard performance metrics of each reader including sensitivity, specificity, precision, and accuracy for interpreting the pre-index screens are summarized in Table 2. The sensitivity of detecting the lesions in pre-index screens is generally lower than the reported sensitivity of mammography (9,10,11), which is expected.

Table 2.

Performance metrics of each reader (R1 – R5) in interpreting pre-index screens in a blinded manner with unblinded consensus read as truth or reference standard.

Performance metrics	R1	R2	R3	R4	R5
Sensitivity	35.3%	54.9%	64.7%	56.9%	64.7%
Specificity	91.3%	56.5%	65.2%	43.5%	26.1%
Precision	75.0%	48.3%	57.9%	42.6%	39.3%
Accuracy	67.5%	55.8%	65.0%	49.2%	42.5%

The individual readers’ sensitivity and specificity in interpreting the pre-index screen are shown in the ROC plot (Figure 2). The performance of two of the readers was close to random chance. Hence, the data were analyzed based on all 5 readers and after excluding the two readers whose performance was close to random chance. Tables 3 and 4 show the distribution of cancer types and lesion types, respectively.

Figure 2.

The individual operating point of each reader in interpreting the pre-index screens in a blinded manner is shown in the ROC plot. The performance of 2 readers (open circles) was similar to random chance. Hence, the data were analyzed by including and excluding these 2 readers. Dashes indicate the identity line.

Table 3.

Distribution of cancer types

Cancer type	Consensus read	Detected by all readers	Detected by 3 readers	Not detected by any reader
All	51	8/51 (16%)	13/51 (25%)	10/51 (20%)
IDC	27/51 (53%)	5/27 (19%)	8/27 (30%)	5/27 (19%)
ILC	6/51 (12%)	1/6 (17%)	2/6 (33%)	0/6 (0%)
DCIS	14/51 (27%)	2/14 (14%)	3/14 (21%)	5/14 (36%)
Other*	4/51 (8%)	0/4 (0%)	0/4 (0%)	0/4 (0%)

^*Other cancer type includes 3 papillary invasive carcinomas and 1 adenosquamous carcinoma.

Table 4.

Distribution of lesion types

Lesion type	Consensus read	Detected by all readers	Detected by 3 readers	Not detected by any reader
All	51	8/51 (16%)	13/51 (25%)	10/51 (20%)
STL*	29/51 (57%)	4/29 (14%)	7/29 (24%)	4/29 (14%)
STL with calcifications	3/51 (6%)	1/3 (33%)	1/3 (33%)	0/3 (0%)
Calcifications only	19/51 (37%)	3/19 (25%)	5/19 (26%)	6/19 (32%)

^*STL represents soft tissue lesions and includes masses, architectural distortions, and asymmetries.

DISCUSSION

Approximately 39 million mammograms are performed annually in the US (15). The high volume of mammograms to be read, in a low-prevalence disease in the population (12), combined with the small and subtle findings to be identified makes diagnosing breast cancer on a mammogram a challenging task. This explains high reader variability and interpretation performance (16,17), which in part explains the missed cancer numbers seen in retrospective studies (4,5,6).

Ikeda et al (6) have documented a missed cancer rate of around 33–58%, which includes interval cancers and false-negatives. While other authors (4,5) also mention similar missed rates, all these studies were unblinded, where the location of the tumor and type of abnormality were known to the reader reviewing the images in hindsight. In our series, on unblinded review, 51/120 (42.5%) had suspicious findings. Unlike Hovda et al (4), who in their review classified lesions as a minimal (32%) versus a missed finding (22%), totaling 54%, we did not make that distinction of a subtle finding versus an obviously missed finding. We used the binary classification and consensus of the 2 reviewing radiologists to determine the presence of an abnormality on a prior study assigned a BIRADS 1 or 2 (20) by the interpreting radiologist. As stressed in earlier literature (14,18) missed cancers do not necessarily mean a radiological error. The real question to ask ourselves is: how often would one have picked the lesion if the images were presented in an unblinded manner, where there is no information to the radiologist as to the presence/absence of cancer? To the best of our knowledge, studies answering that question are sparse. In our study design, we sought to answer this question by a blinded read using five board-certified, fellowship-trained radiologists. Among the 120 pre-index screens, we concluded 51/120 of those screens had a likely finding when interpreted in consensus by two radiologists and in an unblinded manner, which constituted the “ground truth”. Only 8/120 (6.7%) of these lesions were detected by all 5 blinded readers, and most readers (3 out of 5) also identified findings in 13/120 cases (10.8%).

Not just breast density but the background mammographic texture (homogenous versus heterogeneous) makes lesion analysis and identification a difficult task and contributes to errors (10,11, 19). Our study only focused on breast density, not on the mammographic texture, which is not documented in our reports. We did not find any correlation between cancer misses and breast density. Probably given the small number of misses by the majority (3 out of 5) of the radiologists, 13/120 in our study, it was not statistically powered to discern a correlation. Both Hoff et al (4) and Hovda et al (5) also did not observe any correlation between density and misses, though Geiss et al (11) consider dense breast a significant contributor to error in diagnosis.

Breast cancers manifest as masses, asymmetries, architectural distortion, or calcifications on imaging. Though asymmetries and architectural distortion are clear entities as defined in BIRADS Atlas (20), we noted considerable reader variation and inconsistent documentation of asymmetries and architectural distortion in our records. Therefore, for purposes of documentation we clubbed asymmetries, architectural distortion, and masses together as soft tissue lesions (STL). Hovda et al. (4) in their study observed that 82% of documented asymmetries in a prior study presented as masses on subsequent diagnostic images. While Hovda et al (4) and Bird et al (19) noted masses as the finding most missed, Hoff et al (5) noted calcifications as the finding most missed. In our series 29/51(57%) misses were soft tissue lesions (masses, asymmetries and architectural distortion combined) and 19/51 (37%) were calcifications. In the study from Norway by Hovda et al (4) the mean size of the mass missed on a prior screen was 14 mm, which the authors felt should have been called. Our reference to these lesions as STL without clear documentation of the size of a mass was a limitation of our study. Hovda et al (4) noted that cancers with minimal or obvious findings on a prior study had a lower grade compared to cancers that had no imaging finding on a prior. In our series while the stage and type of cancers were uniformly recorded, the pathology grade, a predictor of the aggressiveness was missing in some of the pathology reports limiting our ability to evaluate this parameter. While the study analyzed the missed cancer rate by lesion type, we did not discern any trends.

Multiple reasons have been ascribed to missing a breast cancer diagnosis on image interpretation (6, 11, 13, 14,18, 19, 21). These include technical factors such as poor positioning, motion, and artifacts in the image; patient factors such as body habitus, breast density and size; imaging factors such as lesion type, lesion size, lesion location; and reader factors such as cognitive biases. Cognitive biases, of which a multitude exists, starts with understanding the human decision-making process, essentially consisting of two types: the heuristics or the fast thinking, and the slow and deliberate thinking. These are well described by Lamb et al (14) and Busby et al (21) and are major contributing factors and difficult to assess objectively. The authors suggest several strategies at the individual and institutional levels to overcome these biases. Some of the abovementioned factors contributed to our misses and are illustrated in Figures 3–5.

Figure 3.

A 58-year-old woman had an incompletely imaged irregular mass (arrows in A and B) in the lower inner quadrant (LIQ) at posterior depth of the right breast on her screening mammogram. A: MLO view; B: CC-view. On the CC view, the imaging abnormality is incompletely imaged. C: On the diagnostic spot compression an irregular increased density mass is noted in this region. D: A follow-up ultrasound showed a 13 × 12 × 11 mm irregular, avascular, hypoechoic mass with no posterior features at 4 o’clock and 8cm from the nipple. The lesion was biopsied under ultrasound guidance. E: Post-biopsy mammogram showing biopsy marker. Histopathology indicated a grade 2, ER+, PR−, HER2−, invasive ductal carcinoma. On review of the prior screening (pre-index) mammogram (F: MLO view; G: CC view), a subtle asymmetry is seen on CC view (arrow in G). No abnormality was noted in MLO view (F). The LIQ is the quadrant that is most likely to be incompletely imaged due to positioning challenges. This case illustrates how positioning, lesion location, and reader bias can contribute to misses.

Figure 5.

A 61-year-old woman presented with focal asymmetry (circles in A: CC view and B: MLO view) in the upper outer quadrant at the posterior depth of the right breast on a screening mammogram. The mammograms show heterogeneously dense breasts. C: Ultrasound demonstrates an irregular, shadowing, taller-than-wide mass with some vascularity at 9 o’clock and 6 cm from the nipple corresponding to the mammographic abnormality. D: Ultrasound-guided biopsy shows needle position. E: Post-biopsy MLO view mammogram shows biopsy marker. Histopathology indicated grade 2, ER +, PR+, HER2−, invasive ductal carcinoma with associated grade 2, ductal carcinoma in-situ. On a 2-year prior screening (F: CC view, G: MLO view), the abnormality was not appreciated.

Both the European guidelines for breast cancer screening and the ACR guidelines outlined in the BI-RADS atlas (20,22,23) recommend a regular audit of breast practices. In the United States, while inter-disciplinary team meetings and radiology-pathology correlation conferences are common, regular reviews of all missed and interval cancers are uncommon. Time constraints and expectations of productivity are constraints to this important professional activity. While peer review is a good initial step it does not address this need sufficiently. Unlike in Europe, independent double-read of screening mammograms due to staffing shortages is not currently feasible in the United States. Double-reads have been shown to decrease missed breast cancer rates (17). Hovda et al (4) also mention how discordant independent reads could be a good indicator of potential misses.

With the advent of Artificial Intelligence (AI), we see the future use of AI in screening to reduce missed cancer rates (24,25). Lotter et al (24), using AI software showed improved sensitivity and specificity in the above pre-index population compared to 5 blinded readers, indicating its potential in future applications. In our study design attempting to simulate a screening environment the diagnosis of cancer was delayed by one screening cycle in approximately 11% of the population. Lekanidi et al (26) in their study showed that the estimates of interval cancer rates drop nearly 39% when their readers were blinded. Also, with the transition to digital breast tomosynthesis (27), it would be of interest to determine if the observed rates for missed cancers, constituting a delay in diagnosis, differ from FFDM.

SUMMARY

In summary, among the 120 pre-index screens, all readers detected lesions in 8 screens. Excluding the two readers whose performance was close to random, all the 3 remaining readers detected lesions in 13 pre-index screens. This indicates that there is a delay in diagnosis by at least one cycle from 8/120 (6.7%) to 13/120 (10.8%). There were no observable trends in terms of either the cancer type or the lesion type. Auditing prior screening mammograms in screen-detected cancers can help in identifying the proportion of cases that were missed during interpretation and help in quantifying the delay in breast cancer detection.

Figure 4.

A 63-year-old woman with an upper-central, middle-depth mass in the left breast with obscured borders (circles in A: CC view, B: MLO view). The mass is better seen in spot compression views (C: CC view, D: MLO view). E: A subsequent targeted ultrasound shows a heterogenous, parallel, avascular 17 × 11 × 19 mm, mostly cystic mass with solid components at 12 o’clock and 6 cm from the nipple. F: Post biopsy MLO view mammogram shows the biopsy marker. Histopathology indicated a grade 2, ER+, PR+, HER2+ invasive ductal carcinoma. A 1-year prior screen was negative (G: CC view, F: MLO view). However, the heterogeneous parenchymal texture makes the evaluation of the mammogram difficult. Even though the breast density is scattered fibroglandular, a heterogeneous echotexture can mask focal asymmetries.