Role of Computer-Aided Detection in Very Small Screening Detected Invasive Breast Cancers

Abstract

This study aims to assess computer-aided detection (CAD) performance with full-field digital mammography (FFDM) in very small (equal to or less than 1 cm) invasive breast cancers. Sixty-eight invasive breast cancers less than or equal to 1 cm were retrospectively studied. All cases were detected with FFDM in women aged 49–69 years from our breast cancer screening program. Radiological characteristics of lesions following BI-RADS descriptors were recorded and compared with CAD sensitivity. Age, size, BI-RADS classification, breast density type, histological type of the neoplasm, and role of the CAD were also assessed. Per-study specificity and mass false-positive rate were determined by using 100 normal consecutive studies. Thirty-seven (54.4 %) masses, 17 (25 %) calcifications, 6 (8.8 %) masses with calcifications, 7 (10.3 %) architectural distortions, and 1 asymmetry (1.5 %) were found. CAD showed an overall sensitivity of 86.7 % (masses, 86.5 %; calcifications, 100 %; masses with calcifications, 100 %; and architectural distortion, 57.14 %), CAD failed to detect 9 out of 68 cases: 5 of 37 masses, 3 of 7 architectural distortions, and 1 of 1 asymmetry. Fifteen out of 37 masses were hyperdense, and all of them were detected by CAD. No association was seen among mass morphology or margins and detectability. Per-study specificity and CAD false-positive rate was 26 % and 1.76 false marks per study. In conclusion, CAD shows a high sensitivity and a low specificity. Lesion size, histology, and breast density do not influence sensitivity. Mammographic features, mass density, and thickness of the spicules in architectural distortions do influence.

Introduction

Screening mammography has a sensitivity of 85–90 % in women 49–69 years old, which means that 10–15 % of tumors are missed on mammography [1, 2]. The existence of missed cancers has been documented in various studies. Among cancers that are missed and detected at subsequent screening, 25–40 % can be defined in non-blinded retrospective reviews and considered worthy of work-up or biopsy. Missed cancers are related to several reasons: low disease prevalence, breasts density and complexity, subtle findings, and radiologist fatigue [3].

After the US Food and Drug Administration approved the first commercially available computer-aided detection (CAD) system in June 1998, CAD systems have been increasingly used. In fact, from 2004 to 2008, the use of CAD in screening mammography in the USA increased by 91 %. In 2008, CAD was used in 74 % of all screening mammographic studies in the USA [4]. Several studies in the literature have shown that CAD increases detection rates up to 20 % [5–11]. However, much of these studies were performed by using digitized images which can be a source of variability.

Recently, the study Digital Mammographic Imaging Screening Trial has shown that full-field digital mammography (FFDM) is more accurate than conventional screen-film mammography in women under the age of 50 years, women with dense breasts in mammography, and premenopausal or perimenopausal women [12]. The reported sensitivity of CAD in FFDM varies from 78 to 96 % [10, 13–15] but none of these studies was focused on very small, screening detected, invasive breast cancers.

This study was aimed to evaluate the performance of CAD in very small (less than or equal to 1 cm) invasive breast cancers using FFDM. Mammographic features of the lesion (following BI-RADS descriptors), BI-RADS classification, breast density, size, and histology were assessed in order to determine their influence on CAD.

Materials and Methods

Patient Selection

Institutional review board approval was obtained. We have retrospectively studied 68 invasive breast cancers with a size equal to or lesser than 1 cm diagnosed in women aged 49 to 69 from our population-based breast cancer screening program. Our population-based breast cancer screening program involves three districts in the city of Barcelona (Sants-Montjuic, Eixample esquerre, and Les Corts). Between June 2007 and May 2010, 40,139 women were screened and 235 breast neoplasms were diagnosed (detection rate of 5.85/thousand). Fifty-seven out of 235 (24.2 %) were ductal carcinoma in situ which were excluded because we had planned to assess the sensitivity of CAD in invasive breast cancer. The remaining 178 were invasive cancers. Sixty-eight out of 178 (38.2 %) were invasive cancers that fulfilled the inclusion criteria.

CAD (SecondLook, version 7.2, iCAD, Inc) with FFDM (Senographe 2000 D, General Electric Healthcare) were used. Only screening mammograms were included. Each case had a craniocaudal (CC) and mediolateral oblique (MLO) views of the breast with cancer at the time of cancer diagnosis.

Mammographic and Histological Variables

Mammographic pattern of all visible cancers was recorded. Mammographic lesion types were classified as follows: masses, calcifications, masses with calcifications, architectural distortions, and asymmetries. To describe and classify radiological findings, we adhered to the BI-RADS lexicon. The variables were retrospectively assessed by XB and MV, both subspecialized in breast imaging with more than 10 and 25 years of experience, respectively. When no agreement, a consensus was reached.

Masses were classified according to their morphology (round, oval, lobular, and irregular), margins (well-defined, microlobular, ill defined, obscured, and spiculated), and density (hyper-, iso-, and hypodensity compared with surrounding fibroglandular tissue).

Classification of calcifications was performed according to their morphology, the number of flecks per cluster and the size of the cluster. Regarding the morphology—punctate, amorphous, pleomorphic, linear, branching, or casting. When at least one fine linear or branching calcification was clearly present, the cluster was classified in the last subgroup. Regarding the number of flecks per cluster, less than 10, 10–30, and more than 30 were classified. Regarding the size of the cluster, less than 1 cm, 10–30 mm, and more than 30 mm were classified.

An architectural distortion was defined as the presence of distortion without an associated mass. The total size of the lesion in mammograms and the thickness of the spicules were measured. We classified the thickness of the spicules into thin (1 < mm), medium (1–2 mm), or thick (>2 mm) in order to compare with the role of the CAD.

A focal asymmetry was considered when an asymmetry could not be accurately identified as a true mass (lack of any border).

Mammographic breast density was determined following ACR criteria. Type 1, predominantly fatty breast (<25 % of dense tissue); type 2, scattered fibroglandular densities (25–50 % of dense tissue); type 3, heterogeneously dense breast (50–75 % of dense tissue), and type 4, extremely dense breast (>75 % of dense tissue).

Each lesion was retrospectively assigned a final BI-RADS category as follows: 4a, low suspicion for malignancy; 4b, intermediate suspicion of malignancy; 4c, moderate concern, but not classic for malignancy; and 5, highly suggestive of malignancy.

All patients underwent surgery. Pathological diagnosis was available for all lesions. Size and histological type of the neoplasm were collected. The invasive cancers were histopathologically categorized as follows: invasive ductal carcinoma, lobular carcinoma, and a miscellaneous group including colloid and tubular types.

Following recommendations from European guidelines, we performed double reading with arbitration of discordant readings of each screening mammogram. We have compared cases with a false-negative reading of one of the radiologists with role of the CAD.

CAD Mark Evaluation

The method we used was as follows: we retrospectively sent the raw data of the images to CAD system. The CAD system processed the images and sent them to the PACS. We retrieved the CAD processed images from PACS and checked the marks. CAD operating point was set at “H,” which means maximum sensitivity with poor specificity. CAD used two types of marks: ovals for “masses” (including true masses, asymmetries, and architectural distortions) and rectangles for calcifications. Each CAD mark was classified as either a true-positive or a false-negative mark. True-positive marks indicated correctly a malignant lesion. False-positive marks were all other CAD marks. For cancers appearing as masses, true positive was considered if the center of the lesion fell within the mark. The same marking principle was used for architectural distortions and asymmetries. For calcifications, true positive was considered when the CAD mark involved the majority of the elements of the cluster.

CAD marking according to view was also recorded.

Statistical Analysis

The sensitivity of the CAD system was calculated as the number of cancers correctly marked divided by the total number of cancers. The specificity of the CAD system was calculated as the number of negative studies (mammograms without CAD marks) in 100 consecutive normal four-view studies from our screening program.

The influence of several variables such as mammographic features (following BI-RADS descriptors), BI-RADS classification, breast density, and histology, on CAD performance was evaluated using the Chi-square test. The influence of size on CAD performance was evaluated using “Mann–Whitney U test.”

Results

Sixty-eight cancers were found in women with a median age of 62. We found 37 (54.4 %) masses, 17 (25 %) calcifications, 6 (8.8 %) masses with calcifications, 7 (10.3 %) architectural distortions, and 1 asymmetry (1.5 %). The average tumor diameter of invasive cancers found on the basis of masses was 7.35 mm; calcifications, 5.24 mm; masses with calcifications, 7 mm, and architectural distortions, 6.6 mm.

CAD showed an overall sensitivity of 86.7 %. Sensitivities according to the mammographic appearance are shown in Table 1.

Table 1.

Sensitivity according to mammographic appearance of cancer

Mammographic appearance	Sensitivity (% (detected/total))
Calcifications	100 (17/17)
Masses with calcifications	100 (6/6)
Masses	86.5 (32/37)
Architectural distortions	57.14 (4/7)

CAD detected 59 out of 68 cases. Thirty-five out of 59 were marked in both views, 13 out of 59 were marked in craniocaudal view, and 11 out of 59 were marked in mediolateral oblique view. CAD failed to detect 9 out of 68 cases whose characteristics are shown in Table 2. Five out of 37 were masses, 3 out of 7 were architectural distortions, and 1 out of 1 was an asymmetry.

Table 2.

Characteristics of cancers that were missed by CAD

Mammographic appearance	Breast density	BI-RADS category	Size (mm)	Histology	Morphology	Margins	Density
Mass	Dense	4a	6	IDC	Lobular	Ill-defined	Iso
Mass	Fatty	4b	9	IDC	Irregular	Spiculated	Low
Mass	Fatty	4a	9	IDC	Irregular	Microlobulated	Iso
Mass	Fatty	4a	6	IDC	Irregular	Ill-defined	Iso
Mass	Fatty	4a	8	IDC	Irregular	Spiculated	Iso
Architectural distortion	Dense	4b	1	IDC	Thin spicules
Architectural distortion	Dense	4b	5	IDC	Thin spicules
Architectural distortion	Fatty	4b	7	IDC	Thin spicules
Asymmetry	Fatty	4a	7	IDC

Fatty ACR breast density types 1–2, Dense ACR breast density types 3–4

Five masses were not detected by CAD. No association was seen among mass morphology or margins and ability of the CAD to mark. Fifteen out of 37 masses were hyperdense, and all of them were detected by CAD (p = 0.047). One out of 37 was hypodense and was not detected by CAD. On the other hand, 16 out of 37 masses were marked in both views, 8 were marked in CC view (only one showed spicules), and 8 were marked in MLOBL view.

Calcifications were all detected by CAD. The groups of calcifications measured between few millimeters and several centimeters on the mammographic image, but the actual size of the infiltrating neoplasm in the surgical specimen measured always less than 1 cm. Fourteen out of 17 groups of calcifications showed a cluster larger than 1 cm and all of them contained more than ten flecks. Fourteen of 17 were marked in both projections, 2 out of 17 were marked in craniocaudal view, and 1 out of 17 was marked in mediolateral oblique view. Morphology of the flecks was amorphous in three cases, pleomorphic in nine, and linear or branching in five.

Regarding architectural distortions, CAD failed in three cases in which spicules were thin. Conversely, four cases in which CAD was able to detect, spicules were medium or thick.

Regarding the relationship between CAD and BI-RADS category, CAD failed predominantly in lesions that were labeled as BI-RADS 4a. Likewise, CAD did not fail any case classified as BI-RADS 4c and 5 (Table 3) although this association was not statistically significant.

Table 3.

Relationships among BI-RADS classification, role of CAD, and mammographic appearance

	CAD sensitivity (%)	CAD mistakes	Masses	Calcifications	Masses with calcifications	Architectural distortions	Asymmetries
BI-RADS 4a (n = 26)	80.8	5a	22	2	1	0	1
BI-RADS 4b (n = 23)	83.3	4b	7	7	3	6	0
BI-RADS 4c (n = 7)	100	0	3	2	1	1	0
BI-RADS 5 (n = 12)	100	0	5	6	1	0	0

^aTwo were related to masses only seen on one view

^bThree corresponding to architectural distortions

There were no statistically significant differences when comparing size and breast density with role of the CAD (Table 4). The average size of the lesions detected by CAD was 6.7 mm (SD, 2.4 mm), whereas that of lesions missed by CAD was 6.4 mm (SD, 2.4 mm).

Table 4.

Breast density and CAD

	CAD sensitivity (%)	CAD failed	CAD-marked both views	CAD-marked CC view	CAD-marked MLOBL view
Breast density ACR 1 (n = 17)	88.2	2	11	3	1
Breast density ACR 2 (n = 21)	81	4	9	5	3
Breast density ACR 3 (n = 25)	88	3	13	5	4
Breast density ACR 4 (n = 5)	100	0	2	0	3

Sensitivity ACR 1–2, 84.2 %. Sensitivity ACR 3–4, 90 %

Analysis of histopathological diagnosis—invasive ductal carcinoma was the predominant type accounting for 58 cases. There were four infiltrating lobular carcinoma, five infiltrating tubular carcinoma, and one coloid carcinoma. CAD detected all of these ten carcinomas.

Specificity of the CAD was 26 %. The CAD false-positive rate was 1.76 marks per mammography.

We had 14 false-negative readings in this group of 68 small invasive neoplasms. Six cases were masses, four were calcifications, three were masses with calcifications, and one was an architectural distortion. CAD marked 11 out of 14 of these neoplasms failing to mark two masses and the architectural distortion.

Discussion

Our results using FFDM in very small invasive breast cancers have shown an overall sensitivity of 86.7 % with a specificity of 26 % and 1.76 false-positive marks per case. Our sensitivity for detecting masses has been 86.5 %, which is slightly inferior to that reported in other series with 91–96 % of overall sensitivity [10, 15–17]. These differences are mainly related to the size of the lesions. In our series, the mean size of cancers correctly prompted by CAD was 6.7 mm while in the study by Kim et al. the mean size was 22 mm (sensitivity, 96 %) [10] and in the study by Sadaf et al. it was 12.4 (sensitivity, 91 %). For tumors measuring between 1 and 10 mm, Sadaf et al. reported a sensitivity of 84 % whereas The et al. reported a sensitivity of 89 % for these small tumors. Nevertheless, note should be made that ductal carcinomas in situ had been included in these studies [16, 17]. Other factors that could explain these small differences of sensitivity are the image parameters in each mammography unit, the brand of the CAD system, including software version and operating point, etc. In our study, we decided to set the operating point in “high” because the study was aimed to assess the maximum CAD sensitivity despite this approach may be detrimental to specificity.

CAD has shown to have a greater sensitivity for calcifications than for masses. Our study has shown a 100 % sensitivity of CAD for detecting calcifications and masses with calcifications. This is in accordance to other studies on CAD in FFDM [10, 15, 17]. In view of these results, it could be thought that radiologists can avoid exhaustive search for calcifications, looking instead at areas with calcifications detected by CAD. However, in spite of our high performance, we recommend to be prudent because some studies have shown poor results of CAD in detecting amorphous calcifications [10, 17, 18]. Since our study was focused on invasive carcinoma, most of calcifications were pleomorphic, fine linear or branching calcifications and most clusters had more than 1 cm and contained more than ten elements, which accounts for our highest sensitivity. Four cases in our series had amorphous calcifications, and all of them were detected by CAD.

CAD marked both views in 59 % of detected cases which is in contrast with the study by Vega et al. who reported that 74 % of lesions were marked on both views [19]. In our study, the degree of concordance between views is lesser due to the smaller size of invasive neoplasms in our group of patients, which make them more difficult to be detected by CAD.

Our results have demonstrated that mass density is an important factor influencing the ability of the CAD to detect breast cancer. All of our hyperdense masses were correctly prompted, and one hypodense mass was not marked. These results are in agreement with those of Ellis et al. who reported a difference of 11–25 % (depending on different brands of CAD system) between isodense and hyperdense lesions [20]. Conversely, morphology and margins of the masses do not show any relationship with CAD performance in this group of small neoplasms.

Architectural distortions are the most difficult mammographic pattern to be detected by the radiologist and also by CAD. We have shown a low sensitivity for these lesions (57 %) which is in agreement with the study by Baker et al. who reported a sensitivity of 49 % using older CAD versions [21]. We have analyzed the mammographic and pathological size and the thickness of the spicules to search for associations which could aid in determining what variables are important to detect these elusive lesions. We have not found any relation to the size of the lesion (either mammographic o pathological). However, it is worth noting that three cases with thin spicules were not detected by CAD while the rest of cases showing medium or thick spicules were detected. Although the number of patients in this subgroup is too small to draw conclusions, we think that further research could help to confirm this point.

Our data emphasize the fact that lesions categorized as BI-RADS 4c or five were all detected by CAD. Interestingly, lesions categorized as BI-RADS 4a had the lowest CAD sensitivity which is likely related to the fact that half of CAD mistakes in this subgroup happened in masses only seen in one view (one of them could easily be confused with a lymph node) which likely made the radiologist gave them BI-RADS 4a category.

Sensitivity for cancers presenting as masses for CAD applied to analogical mammograhy has been lower in dense breast compared with non-dense breast. However, recent studies in FFDM have shown that CAD sensitivity for detecting breast cancer is similar in both, dense and non-dense breast. Thus, in the study by Sadaf et al. no differences were found [17]. The study by Yang et al. found a higher, but nonsignificant, difference for detection of mass lesions in fatty breasts [15]. Some other studies have shown that detection rate for malignant lesions tends to decrease while increasing background density [22–24]. Our study has shown that background density does not affect detection. However, most of our patients had non-dense breast because they came from our population-based screening program, which includes mainly postmenopausal women. Moreover, small lesions are easier to depict in non-dense breasts which accounts for the greater proportion of non-dense breast in our patients group. Therefore, no definitive conclusions can be drawn on this subject.

Similar to previous studies, histopathological type of cancer did not have significant influence on CAD performance in our study. Our study shows a high proportion of tubular carcinoma which is thought to be related to the small size of cancers in our study, and all of them were marked by CAD.

The specificity and the rate of false-negative marks is similar to previous studies [19, 20]. The specificity of the CAD is low, which represents a potential drawback for using this technology. On the other hand, sensitivity is quite good and CAD has also been able to correctly mark 11 out of 14 cases that were read as negative by one of our readers. Although this is a promising data, it should be kept in mind that in a hypothetic scenario in which one single reader were assisted by CAD, the radiologist could be distracted by the number of false-positive marks and perhaps wouldn’t recall a true positive mark. Moreover, Taylor et al., reported that 40 % of false-negative readings are due to decision errors (lesions detected but reader decides not to recall the patient for further study) [25].

A limitation of our study is the small sample size. The strengths of our study include assessment of CAD sensitivity based on mammographic features with adherence to BI-RADS descriptors and terminology and the fact that we have only included invasive carcinomas from our population-based screening program.

In view of our results, CAD works well for detecting high-density masses and calcifications but fails to detect some isodense masses and very thin spicules. Moreover, the number of false-negative marks can be distracting for the radiologist. We think that future developments of this technology should attempt to correct these limitations.

In conclusion, CAD shows a high sensitivity and a low specificity in detecting very small invasive breast cancers from our population-based screening program. Lesion size (between 1 and 10 mm), histology, and breast density do not influence sensitivity. Mammographic features turn out to be an important factor affecting CAD sensitivity. Thus, all invasive cancers showing calcifications and all hyperdense masses were correctly prompted. Lesions assigned as BI-RADS 4c-5 and thick spicules in architectural distortions seem to be easier to detect by CAD.