Effect of Breast Density on Computer Aided Detection

Abstract

Purpose: This study was conducted to assess the clinical impact of breast density and density of the lesion’s background on the performance of a computer-aided detection (CAD) system in the detection of breast masses (MA) and microcalcifications (MC). Materials and Methods: A total of 200 screening mammograms interpreted as BI-RADS 1 and suspicious mammograms of 150 patients having a histologically verified malignancy from 1992 to 2000 were selected by using a sampler of tumor cases. Excluding those cases having more than one lesion or a contralateral malignancy attributable to statistical reasons, 127 cases with 127 malignant findings were analyzed with a CAD system (Second Look 5.0, CADx Systems, Inc., Beavercreek, OH). Of the 127 malignant lesions, 56 presented as MC and 101 presented as MA, including 30 cases with both malignant signs. Overall breast density of the mammogram and density of the lesion’s background were determined by two observers in congruence (density a: entirely fatty, density b: scattered fibroglandular tissue, density c: heterogeneously dense, density d: extremely dense). Results: Within the unsuspicious group, 100/200 cases did not have any CAD MA marks and were therefore truly negative (specificity 50%), and 151/200 cases did not have any CAD MC marks (specificity 75.5%). For these 200 cases, the numbers of marks per image were 0.41 and 0.37 (density a), 0.38 and 0.97 (density b), 0.44 and 0.91 (density c), and 0.58 and 0.68 (density d) for MC and MA marks, respectively (Fisher’s t-test: n.s. for MC, p < 0.05 for MA). Malignant lesions were correctly detected in at least one view by the CAD system for 52/56 (92.8%) MC and 91/101 (90.1%) MA. Detection rate versus breast density was: 4/6 (66.7%) and 18/19 (94.7%) (density a), 32/33 (97.0%) and 49/51 (96.1%) (density b), 14/15 (93.3%) and 23/28 (82.1%) (density c), and 2/2 (100%) and 1/3 (33.3%) (density d) for MC and MA, respectively. Detection rate versus the lesion’s background was: 19/21 (90.5%) and 36/38 (94.7%) (density a), 34/36 (94.4%) and 59/62 (95.2%) (density b), 8/9 (88.9%) and 20/24 (83.3%) (density c), and 9/10 (90%) and 4/8 (50%) (density d) for groups 2 and 3, respectively. Detection rates differed significantly for masses in heterogeneously dense and extremely dense tissue (overall or lesion’s background) versus all other densities (Fisher’s t-test: p < 0.05). A significantly lowered FP rate for masses was found on mammograms of entirely fatty tissue. Conclusion: Overall breast density and density at a lesion’s background do not appear to have a significant effect on CAD sensitivity or specificity for MC. CAD sensitivity for MA may be lowered in cases with heterogeneously and extremely dense breasts, and CAD specificity for MA is highest in cases with extremely fatty breasts. The effects of overall breast density and density of a lesion’s background appear to be similar.

Introduction

Mammography is a well-established method for the early detection of breast cancer. Because of the very high interobserver variability, breast cancer detection rates can be improved by up to 15% using a second reader.^1–5 Alternatively, computer-aided detection (CAD) systems are being evaluated if they are equivalent to a second reader.^6–9 Efforts were made as early as 1967 to develop a CAD system for mammography.¹⁰ These systems are designed to help radiologists make earlier and more accurate detection of such features as suspicious masses (MAs) and microcalcifications (MCs) as well as architectural distortions during screening mammography. Funovics et al.¹¹ showed that the sensitivity for breast cancer detection increases significantly when a radiologist uses a CAD system. Recently published studies critically discuss the option of CAD systems as a primary diagnostic tool rather than as an aid for a radiologist,^12,13 and negate their usage in screening populations.⁹

Although the number of false positive (FP) findings of CAD systems is still discussed as a major limitation of CAD,¹⁴ Marx et al.¹⁵ demonstrated, that the use of CAD does not induce an increased recall rate or an increased number of (unnecessary) biopsies.

On the other hand, it is reported that CAD systems detect a relevant number of interval cancers.^16,17

Significant differences in case selection criteria may affect the measured performance of a CAD system, which may explain the differences in sensitivity values obtained in previous studies.^18–20

To appropriately estimate the potential impact of a CAD, it is important to analyze the factors that influence tumor detectability by a radiologist and a CAD. For example, the histopathological features and the lesion size of the breast cancer influence radiologist sensitivity^18–20 and the tumor detection capability of a CAD system,²¹ especially extremely large cancers of rare histology (i.e., cancers >4 cm in size and mucinoid cancers).

To our knowledge, only one study has analyzed the potential effect of breast density on CAD detection,²² which is known to critically limit mammographic evaluation by radiologists, especially in younger patients.²³ One of these studies²² showed that increasing breast density reduces cancer detection by CAD without affecting the CAD false positive rate. This study used an early software version and did not show the effect of breast density on CAD performance with malignant masses and microcalcifications separately. No study to date has addressed the impact of the lesion’s background density to the detection of masses by CAD.

Therefore this study focuses on answering the following questions:

1. Is CAD performance in malignant microcalcifications and/or masses influenced by the overall breast density, using the ACR BI-RADS^® lexicon for overall breast composition?

2. Is CAD performance in malignant microcalcifications and/or masses influenced by the background density of the suspicious lesion?

3. Does the total number of false positive findings per image depend on the density of the breast?

Patients and Methods

Study Concept and Cases

A retrospective analysis of mammograms from 200 unsuspicious cases with mammographic follow-up of at least two years and from 150 patients having a histologically proven breast cancers was performed. In effect, every fifth biopsy-proven cancer case from September 1992 to March 2000 implemented in the tumor case sampler of our breast department was selected. All mammograms showing tumor-induced changes that were histologically proven in the department and that had led to the diagnosis of breast cancer had been stored separately in the internal tumor case sampler. Thus a wide range of tumor sizes was included in the study without any preselection. Cases that were first detected in external mammography centers were not included into the tumor sampler (and therefore excluded from the study) to avoid influences resulting from technical differences. All cases with more than one mammographically visible suspicious lesion per image and those having contralateral cancers were excluded from the study (for statistical reasons) to eliminate a possible bias that could arise from similar mammographic features of multifocal or contralateral cancer originating from a single woman. No other form of preselection was performed. Overall, 127 patients having a malignant finding were included. All tumor sizes and histologies were included, and all malignant lesions were surgically verified.

After due consideration, the ethical board gave its consent for the study.

All mammographic examinations were conducted with the Mammodiagnost UC (Philips, Netherlands) or Senographe DMR (GE Medical Systems, USA) between September 1992 and March 2000. Each mammographic examination consisted of two images: the craniocaudal (CC) and mediolateral oblique (MLO) view of the right or left breast presenting with no (group 1) or one suspicious lesion (groups 2 and 3).

The local ethical board granted approval for the study protocol.

Histopathological procedures revealed 127 malignant lesions.

Seventy-one mammographically suspicious lesions were described due to a mass, 26 due to microcalcifications, and 30 showed both signs of malignancy, yielding an overall number of 56 suspicious microcalcifications (group 2) and 101 suspicious masses (group 3). All mammograms were routinely double-read by at least two experienced radiologists.

Breast Density Determination

Breast density assessments were performed in consensus by two experienced radiologists, who were blinded to the histological outcome and CAD performance. The cases were divided into four groups according to the density patterns used by the American College of Radiology’s Breast Imaging Reporting and Data System (BI-RADS^®):

• Density a: entirely fatty

• Density b: scattered fibroglandular densities

• Density c: heterogeneously dense

• Density d: extremely dense

Adapted to these patterns, the lesion’s background density was additionally determined in consensus, taking into account the direct surrounding area by analyzing a square of approximately 2 cm² around the lesion.

CAD Performance

The mammograms were processed in two views by the CAD system (Second Look Version 5.0, CADx Systems, Inc., Beavercreek, OH).

The location on the CAD report corresponding to the mammographically detected and histopathologically confirmed cancer lesion was analyzed by two radiologists in consensus to determine if the CAD system correctly marked the lesion. The lesion was scored as true positive (TP) if the CAD system marked the correct lesion type (microcalcifications or mass) in at least one of the two views. All CAD marks that were not located on the suspicious lesion were scored as false positive (FP) to determine the number of FP marks per image.

Statistical Analysis

Chi-square tests and Fisher–Freeman–Holton test were used to assess the significant differences in CAD detection rates versus overall breast density and the lesion background density. The datasets of microcalcifications (group 2) and masses (group 3) were analyzed independently from each other. A p-value less than 0.05 was judged as significant.

Results

Screening Cases (Group 1)

In this group, 100 out of 200 cases did not have any CAD mass marks in both images and were therefore judged as truly negative (specificity 50%). One hundred and fifty-one cases out of 200 did not have any CAD microcalcification markers on any of the two images (specificity 75.5%).

The mean number of false positive per case was observed to be 0.83 for masses and 0.42 for microcalcifications (0.42 and 0.21 per image, respectively), suggesting a mean of 1.25 false positive markers per case.

For these 200 unsuspicious cases, the number of false positive marks of microcalcifications per image was 0.41 within density group a, 0.38 within density group b, 0.44 within density group c, and 0.58 within density group d. These false positive marker rates were not statistically different, although a slight tendency toward an increase of FP within increasing density could be obtained.

The number of falsely positive markers for masses set by the CAD system was 0.37 for density group a, 0.97 for density group b, 0.91 for density group c, and 0.68 for density group d, showing a statistically significant difference between group 1 and the other ACR groups (p < 0.05, Fisher’s t-test; see Table 1).

Table 1.

False positive rate of markers for masses and microcalcifications, given in relation to the ACR density criteria, screening group

all cases, ACR density	Falsely positive mass markers (mean)	Falsely positive microcalcification markers (mean)	Mean number of FP markers (per case)
1	0.37	0.41	0.78
2	0.97	0.38	1.35
3	0.91	0.44	1.35
4	0.68	0.58	1.26
Total	0.83	0.42	1.25

CAD Tumor Detection Rate (Groups 2 and 3)

Malignant lesions were correctly detected in at least one view by the CAD system for 52 out of 56 (92.8%) malignant microcalcifications and 91 out of 101 (90.1%) malignant masses. Out of these detected malignancies, 57/101 masses were correctly detected in both views and 34 in one view, respectively, whereas 10 malignant masses were not marked. Out of the 56 microcalcifications, 21 were marked in one view and 31 in both views, whereas four lesions remained without any marking (Table 2). The overall number of false positive markers in this subpopulation was 0.86 markers per mammogram (0.43 per image). In the mean, 0.51 falsely positive density markers per case (0.26 per image) and 0.35 falsely positive markers of microcalcifications per case (0.18 per image) were set.

Table 2.

Sensitivity and specificity of CAD in one and in both views

	Sensitivity		Sensitivity, both views		Sensitivity, one view
	Absolute	Percentage	Absolute	Percentage	Absolute	Percentage
MA	91/101	90.1	57/101	56.4	34/101	33.7
MC	52/56	92.9	31/56	55.4	21/56	37.5

Overall Breast Density

CAD detection of malignant microcalcifications (group 2) versus overall breast density was 4/6 (66.7%) in entirely fatty tissue, 32/33 (97.0%) in scattered fibroglandular tissue, 14/15 (93.3%) in heterogeneously dense tissue, and 2/2 (100%) in extremely dense tissue.

There was no statistically significant difference in CAD detection of malignant microcalcifications versus overall breast density. The CAD detection rate of malignant masses (group 3) was 18/19 in entirely fatty tissue (94.7%), 49/51 in scattered fibroglandular tissue (96.1%), 23/28 in heterogeneously dense tissue (82.1%), and 1/3 (33.3%) in extremely dense tissue. The detection rate of malignant masses was statistically significantly lower in heterogeneously and extremely dense breasts versus the other breast density groups (p < 0.05).

The FP rate for masses and microcalcifications is given separately for each density in Table 1, showing a significantly lowered falsely positive rate within the ACR-1 group for both microcalcifications and mass markers.

The ratio of falsely positive markers, when focusing on cancer cases exclusively, is given in Table 3. Differences revealed a statistically significant value having a lowering of false positive findings in ACR group 1. The highest numbers of falsely positive mass markers are observed in ACR group 2.

Table 3.

Falsely positive markers for masses and microcalcifications and its relation to breast density according to ACR criteria, cases being histologically verified as malignant (groups 2 and 3)

Malignant cases, ACR density	Falsely positive mass markers (mean)	Falsely positive microcalcification markers (mean)	Mean number of FP markers (per case)
1	0.30	0.15	0.45
2	0.58	0.42	1.00
3	0.51	0.38	0.89
4	0.50	–	0.50
Total	0.51	0.35	0.86

Background Density of the Suspicious Lesion

Detection rate of malignant microcalcifications versus the lesion’s background was 4/6 (66.7%) in density group 1, 32/33 (97.0%) in density group 2, 14/15 (93.3%) in density group 3, and 2/2 (100%) in density group 4. The results suggest that there is no statistically significant influence of the lesion’s background density on the detection rate of malignant microcalcifications by CAD (Table 4).

Table 4.

Detection rate for masses and microcalcifications and its relation to breast density according to ACR criteria, cases being histologically verified as malignant (groups 2 and 3)

Malignant cases, ACR density	Detection rate, microcalcifications	Detection rate, masses
1	4/6 (66.7%)	18/19 (94.7%)
2	32/33 (97.0%)	49/51 (96.1%)
3	14/15 (93.3%)	23/28 (82.1%)
4	2/2 (100%)	1/3 (33.3%)

For malignant masses, the CAD detection rate was 18/19 (94.7%) in density group a, 48/51 (94.1%) in density group b, 24/28 (85.7%) in density group c, and 1/3 (33%) in density group d, yielding a significantly lowered detection rate of masses with an extremely dense background density. The distribution of the mean number of falsely positive set markers in relation to the lesion’s background is given in Table 5, and the detection rates in Table 6.

Table 5.

Falsely positive markers for masses and micrcalcifications and its relation to density of the background of the lesion according to ACR criteria, cases being histologically verified as malignant (groups 2 and 3)

Malignant cases, ACR density	Falsely positive mass markers (mean)	Falsely positive microcalcification markers (mean)	Mean number of FP markers (per case)
1	0.39	0.28	0.67
2	0.58	0.43	1.01
3	0.52	0.16	0.68
4	0.50	0.83	1.33
Total	0.51	0.35	0.86

Table 6.

Detection rate for masses and microcalcifications and its relation to density of the background of the lesion according to ACR criteria, cases being histologically verified as malignant (groups 2 and 3)

Malignant cases, ACR density	Detection rate, microcalcifications	Detection rate, masses
1	19/21 (90.5%)	36/38 (94.7%)
2	34/36 (94.4%)	59/62 (95.2%)
3	8/9 (88.9%)	20/24 (83.3%)
4	9/10 (90.0%)	4/8 (50.0%)

The ratio of correct detection of the malignant mass in both views by markers depends on the density of the breasts, as given in Table 7. It decreases with increasing breast density. In contrast to this, the correct detection of microcalcifications in both views is not influenced by the breast density.

Table 7.

Ratio of correct detection of malignant masses and microcalcifications in both views in relation to breast density

Malignant cases ACR density	Detected MA in both views	Detected MC in both views
1	15/19 (78.9%)	3/6 (50.0%)
2	27/51 (52.9%)	18/33 (54.5%)
3	14/28 (50.0%)	8/15 (53.3%)
4	1/3 (33.3%)	2/2 (100%)
Total	57/101 (56.4%)	31/56 (55.4%)

Discussion

Overall Tumor Detection Rate

A CAD system assists the radiologist in the early detection of breast cancer by highlighting suspicious areas. A true positive mark by the CAD system is defined as the correct identification of the cancer in at least one of two views. The Second Look CAD system showed a sensitivity of 92.1% for MCs and 90.2% for MAs. Both values are promising when used in this subpopulation.

The main application area of CAD is in screening. Taking this into account, CAD performance could still be improved, particularly in its rather low specificity values (50% specificity for masses and 75.5% for microcalcifications). However, the significant decrease of false positive mass markers with Second Look version 5.0 versus 3.5, in association with a slight increase of sensitivity, documents the fast developments in this area.²⁴

Several studies of the tumor detection rates of various CAD systems have been published.^20,25–28 The case selection protocols from these studies differ considerably. As case selection has been shown to substantially affect the evaluation of a CAD system performance (including its sensitivity), it is not possible to compare the results of these previous studies.²⁹ However, the overall detection rate in these studies was about 90%, giving a verifiable advantage to the radiologist’s accuracy,^27,30 and the recall rate as well as the biopsy rate decreases with the use of CAD in addition to second reading.¹⁵ Some studies suggest a small increase in recall rate because of the additional use of CAD systems.³¹

Breast Density

Ho and Lam documented a significant influence of overall breast density toward the detection rate of malignancies by using one of the earliest available software versions of the Second Look system.²² The discrepancy in our findings, in which only ACR-3 and -4 breasts have reduced detectability of masses, can be explained by software improvements. Furthermore, in the paper by Ho and Lam, it was not stated whether the number of suspicious lesions per mammogram exceeded one per image (which might effect a higher FN number and limit the statistical analysis) and no separate analysis of microcalcifications and masses was performed. In our study, however, it could be shown that the extremely dense breast tissue reduced the CAD sensitivity for malignant masses, whereas the detection of malignant microcalcifications was not affected by the breast density. The exclusion of lesions larger than 2.5 cm in size by Ho and Lam was reasonable with the software version they used. The currently available software versions, however, allow the detection of even large masses, although the sensitivity for such lesions is still lower.²¹

Our study is more comparable to the results published by Birdwell et al.¹⁶ As with this study (and in contrast to the study of Ho and Lam), no exclusion of cases was performed because of the disagreement in classifying the breast density of the mammograms. In contrast to the study by Birdwell et al., we tested radiologically detected malignancies and screening cases. The fact that extremely dense breasts had a significantly lowered detection rate for masses, but not a significantly increased false positive rate, supports the current management of these cases: using x-ray allows a reliable exclusion of malignant microcalcifications, but neither the radiologist nor the CAD system has a sufficient sensitivity to detect malignant masses in extremely dense breasts. Therefore additional diagnostic techniques, including ultrasound and in special cases contrast-enhanced MRI, are required to allow an accurate diagnosis of malignant masses in extremely dense breasts.

A limitation of this study is that both extremes of breast density, entirely fatty tissue, as well as extremely dense breast tissue, were quite rare compared to the intermediate ACR grades 2 and 3, although distribution in this study is consistent with daily practice and the case composition of other studies addressing this issue. It should be indicated that although these results suggest that there is an effect, a larger sample size would be beneficial to underline the results by statistical tests. The role of this issue in clinical routine, however, is rather small because ACR 4 densities are rare compared with the other density categories.

The usage of other modalities of a semiquantitative measurement of breast density might be of further benefit. These other options, however, are not common in our department and so the interobserver variability might increase by the use of these rather uncommon density calculations. Furthermore, the ACR criteria are widely accepted and used in the daily routine of breast diagnosis.

Based on our study, it can be assumed that perifocal density has a similar influence on the detection of breast cancer as overall density.

Furthermore, not only the detection rate of malignant masses, but also the mean number of falsely set mass markers are influenced by breast density documenting the lowest values in the entirely fatty breast, whereas all other breast densities were characterized by similar rates of false positive findings.

In concordance with other publications, the overall number of false positive findings is significantly lower in cases featuring a malignant finding. This can be explained by the characteristics of CAD systems, which usually limit the maximum number of markers to be set on a mammogram.¹⁴

Conclusion

The detection of malignant microcalcifications is not affected by the density of the breast nor by the density of the background surrounding the suspicious lesion. In contrast, malignant masses in heterogeneously dense and extremely dense breasts are less detectable, which raises the question on the usefulness of CAD in this population group. The false positive rates of masses, both in the screening group and in the cancer group, are affected by breast density, suggesting a lowering of falsely positive findings in ACR 1 breasts.

The false positive rate, although lower when compared to results obtained with older software versions, still reaches a relevant level, and differs considerably among the groups (being lower in the cancer groups). These values, although decreasing, might still affect the use of CAD in screening conditions. The high tumor detection rate of both masses and microcalcifications, however, underlines the usefulness of CAD in clinical mammography.