Microcalcification detectability in breast CT images using CNN observers

Abstract

Background:

Breast computed tomography (CT) is an emerging breast imaging modality, and ongoing developments aim to improve breast CT’s ability to detect microcalcifications. To understand the effects of different parameters on microcalcification detectability, a virtual clinical trial study was conducted using hybrid images and convolutional neural network (CNN)-based model observers. Mathematically generated microcalcifications were embedded into breast CT data sets acquired at our institution, and parameters related to calcification size, calcification contrast, cluster diameter, cluster density, and image display method (i.e., single slices, slice averaging, and maximum-intensity projections) were evaluated for their influence on microcalcification detectability.

Purpose:

To investigate the individual effects and the interplay of parameters affecting microcalcification detectability in breast CT.

Methods:

Spherical microcalcifications of varying diameters (0.04, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40 mm) and native intensities were computer simulated to portray the partial volume effects of the imaging system. Calcifications were mathematically embedded into 109 patient breast CT volume data sets as individual calcifications or as clusters of calcifications. Six numbers of calcifications (1, 3, 5, 7, 10, 15) distributed within six cluster diameters (1, 3, 5, 6, 8, 10 mm) were simulated to study the effect of cluster density. To study the role of image display method, 2D regions of interest (ROIs) and 3D volumes of interest (VOIs) were generated using single slice extraction, slice averaging, and maximum-intensity projection (MIP). 2D and 3D CNNs were trained on the ROIs and VOIs, and receiver operating characteristic (ROC) curve analysis was used to evaluate detection performance. The area under the ROC curve (AUC) was used as the primary performance metric.

Results:

Detection performance decreased with increasing section thickness, and peak detection performance occurred using the native section thickness (0.2 mm) and MIP display. The MIP display method, despite using a single slice, yielded comparable performance to the native section thickness, which employed 50 slices. Reduction in slices did not sacrifice detection accuracy and provided significant computational advantages over multi-slice image volumes. Larger cluster diameters resulted in reduced overall detectability, while smaller cluster diameters led to increased detectability. Additionally, we observed that the presence of more calcifications within a cluster improved the overall detectability, while fewer calcifications decreased it.

Conclusions:

As breast CT is still a relatively new breast imaging modality, there is an ongoing need to identify optimal imaging protocols. This work demonstrated the utility of MIP presentation for displaying image volumes containing microcalcification clusters. It is likely that human observers may also benefit from viewing MIPs compared to individual slices. The results of this investigation begin to elucidate how model observers interact with microcalcification clusters in a 3D volume, and will be useful for future studies investigating a broader set of parameters related to breast CT.

1. INTRODUCTION

Microcalcifications are key indicators of potential early-stage breast cancer which can manifest in various morphologies and distributions^1-3. The American College of Radiology has released a BI-RADS report⁴ on morphological patterns of microcalcifications commonly seen in mammography along with their potential for malignancy. Large rod-like calcifications, for instance, are typically benign and do not necessitate further diagnostic imaging or biopsy. Clustered microcalcifications – dense groupings of small (< 1 mm) calcifications within a small region – are likely to be malignant^5,6, and generally necessitate biopsy. During breast cancer screening exams, it is crucial to accurately detect and characterize microcalcifications.

Digital mammography has historically been used for breast cancer screening, and its capabilities for imaging with high resolution have been valuable for microcalcification detection^7,8. Mammography’s sensitivity substantially decreases, however, when used to image patients with dense breasts due to the superposition of fibroglandular tissue which can obscure lesions. In recent years, breast computed tomography (CT) has emerged as a promising alternative imaging tool^9,10, with the key advantage of accessing fibroglandular anatomy inside the breast without the superposition of neighboring tissue. Initial clinical studies⁹ suggested that breast CT is better than mammography at detecting mass lesions, but that mammography is better than breast CT at detecting microcalcifications. Since this study was published, our laboratory at UC Davis has developed a higher-resolution breast CT scanner which achieves nearly four times the spatial resolution than that of the earlier generation scanner based on their system modulation transfer functions¹¹. This scanner’s ability to detect microcalcifications is under evaluation, and protocols for optimizing the scanner for microcalcification detection are being developed. Ideally, such optimization would involve extensive clinical trials with human observers to evaluate image quality, diagnostic performance, and the impact of various imaging parameters. However, the challenges of conducting large-scale clinical trials, such as time and cost limitations, necessitate the exploration of alternative approaches.

Simulation studies have been proposed as an alternative to clinical studies, where synthetic images are generated using phantom imaging^12-14 or computer simulations^15-19. These studies are also called “virtual clinical trials” ^20-22. In this work, hybrid images are computer simulated, where mathematically generated microcalcifications are embedded into the reconstructed volumes of patient breast CT images acquired at our institution using the high-resolution breast CT scanner. This approach enables the investigation of many lesion-related parameters (e.g., size) while preserving crucial patient- (e.g., breast density) and imaging-related parameters (e.g., resolution). Hybrid images can also be generated by inserting synthetic lesions in the projection domain; this approach has been explored in both mammography²³ and digital breast tomosynthesis²⁴. While there are advantages to this approach, such as a more accurate simulation of certain imaging processes (e.g. beam hardening), it is challenging to implement this approach for breast CT due to the computational burden of reconstructing image volumes for every iteration of lesion insertion. Thus, we insert lesions into the reconstructed image domain while simulating key imaging processes such as partial volume and apodization kernel.

To detect the simulated microcalcifications, a convolutional neural network (CNN) is used in lieu of a human observer. CNN-based model observers are thought to approximate human visual perception^25-27, providing an efficient and reproducible means of quantifying microcalcification detectability across several important parameters. CNNs have shown to be useful and versatile across imaging contexts including breast CT^27,28. Traditional model observers, such as the pre-whitened matched filter or Channelized Hotelling observer, are not explored in this study, though they are commonly used for observer studies. The field has generally found that CNN-based model observers can outperform mathematical model observers, and this includes recent studies by Baek et al.^26,28,29 using synthetic breast CT images with simulated anatomical background, and Lyu et al.³⁰ using hybrid breast CT images with patient anatomical background. These studies concluded that CNN’s capture more diagnostic information from breast CT images than mathematical filters, and therefore may be more useful for conducting optimal performance studies. As we are interested in optimizing breast CT parameters for microcalcification detection, we focus on CNN-based observers in this work. In this study, 2D and 3D CNN models are used to detect individual microcalcifications and microcalcification clusters embedded in 109 patient breast CT data sets, and parameters related to calcification size, calcification contrast, cluster diameter, cluster density, and image display method (i.e., single slices, slice averaging, and maximum-intensity projections) are studied.

2. METHODS

2.1. Breast CT system

Four generations of pendant geometry, cone-beam breast CT scanners have been designed and fabricated in our laboratory over the past two decades. The fourth-generation prototype scanner follows a similar design as previously reported scanners⁹ but achieves nearly four times the spatial resolution of the early scanners based upon modulation transfer function (MTF) analysis¹¹. This is due to the combination of a pulsed X-ray source, a smaller focal spot, and a higher resolution flat-panel detector. The x-ray source contains a rotating tungsten anode x-ray tube with a nominal focal spot of 0.3 × 0.3 mm and a 14kW x-ray generator (CMP200SE, CPI, Ontario, Canada). The flat-panel detector (Dexela 2923M, Varex, Salt Lake City, UT) features a 0.45 mm thick thallium-activated structured cesium iodide (CsI:T1) scintillator coupled to CMOS detector elements. The gantry rotates in the horizontal plane during image acquisition, using a 60 kV x-ray beam with 0.20 mm copper filtration. A total of 500 projection images are acquired in ~11 s. Acquired projection images are reconstructed using the Feldkamp algorithm³¹ with a Shepp-Logan apodization kernel cutting off at 1.5 times Nyquist.

2.2. Patient images

An IRB-approved clinical trial was conducted at UC Davis Medical Center evaluating the fourth-generation breast CT system as a tool for breast cancer screening and for the diagnostic breast exam. Patients with suspicious lesions (BI-RADS 4 or 5) based on screening breast imaging (mammography or tomosynthesis) were eligible to participate in the study. The scanning protocol involved four sequential scans of the 1) unaffected breast prior to contrast-injection, 2) affected breast prior to contrast injection, 3) affected breast after contrast injection, and 4) unaffected breast after contrast injection. After scans were acquired, patients underwent breast biopsy on areas of suspicion based upon standard of care which provided the ground truth diagnosis. To date, 58 women have been scanned on the Doheny breast CT system resulting in 222 breast CT volume data sets. Contrast injection is advantageous for visualizing malignant solid tumors due to local perfusion of contrast agent through “leaky” angiogenetic vessels, and this contrast greatly improves lesion conspicuity³². Because this study focuses on microcalcification detectability, only pre-contrast volume data sets were used to avoid contrast enhancement in and around microcalcifications. In total, 109 pre-contrast volume data sets were selected for this study to be used as anatomical background for mathematical microcalcification insertion. Each volume data set contained 800-900 reconstructed slices (1024 × 1024 matrix size) with isotropic voxel dimensions of 0.2 mm.

2.3. Microcalcification simulation

Spherical microcalcification profiles were generated using methodology developed by Hernandez et al.¹³ First, a single high-resolution microcalcification profile was generated by inserting a sphere of intensity $H$ in the center of a 330 × 330 × 330 matrix with an isotropic voxel dimension of 0.01 mm. Then, the volume was blurred in the $xy$ plane by (i) converting each coronal slice to the frequency domain using the 2D Fourier transform, (ii) multiplying the slice by the 2D coronal plane MTF measured on the breast CT system, and (iii) converting the blurred slice to the spatial domain using the 2D inverse Fourier transform. The volume was then blurred in the $z$ dimension by (i) converting each vector in the $z$-direction to the frequency domain using the 1D Fourier transform, (ii) multiplying the vector by the 1D $z$-direction MTF measured on the breast CT system, and (iii) converting the blurred vector to the spatial domain using the 1D inverse Fourier transform. The MTFs were not assumed to be equal in the $xy$ and $z$ directions due to the role of the Shepp-Logan reconstruction kernel in the $xy$ dimensions but not the $z$ dimension. The high-resolution profile was then down sampled to match the voxel dimensions of the breast CT system (0.2 mm), resulting in a 17 × 17 × 17 matrix, $m(x,y,z)$. The microcalcification profile generation process is illustrated in Figure 1 for a 0.2 mm diameter calcification.

Figure 1:

Simulation of a 0.2 mm microcalcification profile. As a (a) mathematically generated high-resolution microcalcification undergoes (b) XYZ MTF blurring and (c) down sampling, the microcalcification’s edges are blurred and signal is attenuated.

In addition to edge blurring, the blurring procedure resulted in grayscale reduction due to partial volume effects. Let the intensity of a simulated high-resolution calcification, $H$, be 1000 HU, and the intensity of the background volume, $B$, be −200 HU, as shown in Figure 1. Let $y$ be the peak intensity in HU of the calcification after the blurring procedure described above. $y$ was computed for a range of computer-generated calcification diameters between 0 and 1 mm. A plot of peak intensity as a function of calcification diameter in mm, $D_{calc}$, is shown in Figure 2. Based on Figure 2, it was found that intensity reduction resulting from the blurring procedure can be modeled using a logistic function for any input intensity in HU, $H$, calcification diameter in mm, $D_{calc}$, and background intensity in HU, $B$, using Equation 1:

$$y(D_{calc},B,H)=B+\frac{H-B}{1+e^{-\left(\frac{D_{calc}-.52}{.15}\right)}}$$ (1)

Figure 2:

Peak intensity in HU of calcifications after blurring as a function of calcification diameter, $D_{calc}$, showing the partial-volume effects in the system. The grayscale values of the initial high-resolution calcification and background were 1000 HU and −200 HU, respectively.

Equation 1 is derived from fitting the data points in Figure 2 with a logistic function (R² = 0.99). The parameters in the Equation 1 (i.e. 0.52 and 0.15) are unique to our imaging system but can be derived for any imaging system, and illustrate the key principle of partial volume that occurs in any imaging system: objects smaller than the spatial resolution of the imaging system experience blurring and signal attenuation when imaged. In Figure 2, as the calcification diameter approaches zero, the peak intensity of the blurred calcification approaches the intensity of the background, $B$, and as calcification diameter exceeds 1 mm, the peak intensity of the blurred calcification approaches the native intensity, $H$. In this study, $B$ was computed for each simulated microcalcification as the median intensity of a 7 × 7 × 7 subvolume of anatomical background surrounding the location of microcalcification insertion.

$H$ was estimated empirically by applying Equation 1 to select calcifications found in our clinical patient data sets. Two large calcifications in two separate patients were identified using the patients’ screening exam radiology reports and verified using the corresponding pathology reports. Only large, pathology-confirmed calcifications were used for determining $D_{calc}$ in order to maximize accuracy in delineating calcification edges. Custom-built breast CT viewing software was used to measure the diameter and peak intensity for each calcification. The median intensity of the tissue surrounding the calcifications was also measured. These variables were applied to Equation 1 and resulted in native intensity values, $H$, of 361 HU and 768 HU. The range of native intensity values corresponds to varying densities and compositions of individual calcifications in our clinical data sets^1,33.

2.4. Hybrid image generation

For every breast CT volume, lesion-present and lesion-absent region of interest (ROI) patches were generated at random locations within the breast. For each ROI, a lesion center was computer generated using a random number generator and kept if the surrounding volume $V[i,j,k]$ was fully contained within the patient’s breast and did not contain skin. Otherwise, a new lesion center was computer generated. For viable lesion centers, the surrounding volume $V[i,j,k]$ served as the anatomical background for mathematical microcalcification insertion. This method of lesion center selection was consistent for lesion-present and lesion-absent patch generation. Random lesion center selection allowed for datasets to contain a variety of anatomical backgrounds which enabled model generalizability. The lesion centers were saved and used for every iteration of ROI generation in this study to reduce inter-dataset variability such that differences in model observer performance could be accredited to the parameter of interest rather than differences in the local anatomical background. The selection of training ROIs were different than testing ROIs because different patient data sets were held-out for testing as explained in the following section.

2.4.1. 2D image generation

First, 2D hybrid image ROIs were generated such that each ROI contained a single microcalcification of known diameter at the center of the ROI. This setting can be described as signal-known-exactly (SKE), location-known-exactly (LKE). SKE, LKE tasks can be used to establish baselines for evaluating an observer’s detection performance prior to experimenting with unknown or uncertain signals and locations.³⁴

Let $V[i,j,k]$ be a 120 × 120 × 120 anatomical background volume, $M[i,j,k]$ be a volume of the same size with a blurred microcalcification profile $m[x,y,z]$ in the center, and zeros elsewhere, and $V_{sim}[i,j,k]$ be the simulated volume with the inserted calcification. $M[i,j,k]$ is scaled such that when it is added to $V[i,j,k]$, the peak intensity of $V_{sim}[i,j,k]$ equals $y(D_{calc},B,H)$ from Equation 1. This process simulates the partial volume-related blurring of a calcification with surrounding voxels. Let $[i_{max},j_{max},k_{max}]$ be the indices of the peak intensity voxel in $M[i,j,k]$. The simulated volume $V_{sim}[i,j,k]$ is then defined as:

$$V_{sim}[i,j,k]=V[i,j,k]+\left(\frac{y(D_{calc},B,H)-V[i_{max},j_{max},k_{max}]}{M[i_{max},j_{max},k_{max}]}\times M[i,j,k]\right)$$ (2)

$V_{sim}[i,j,k]$ was generated at each lesion center and displayed in 2D by either extracting the center slice of $V_{sim}[i,j,k]$ in the axial view plane or by slice averaging across adjacent slices to model thicker sections. Similarly, lesion-absent ROIs were generated and displayed in 2D by either extracting the center slice of $V[i,j,k]$ in the axial view plane or by slice averaging across adjacent slices. The resulting 2D ROIs had dimensions of 120 × 120 × 1. Five microcalcification diameters $D_{calc}$ (0.04, 0.10, 0.15, 0.20, 0.40 mm), three input intensities $H$ (361, 565, 768 HU), and five section thicknesses (0.2, 0.6, 1.0, 3.0, 11.0 mm) were studied. The equivalent number of slices $n_{slices}$ for each section thickness was 1, 3, 5, 15, and 55 slices, respectively.

Out of the 109 total breast CT volume data sets, 99 data sets (N = 99, ~90%) were used to generate training ROIs for training the model observer. For each volume data set, 200 lesion-present ROIs and 200 lesion-absent ROIs were generated from different lesion centers resulting in 39,600 total training ROIs. 10% of the training ROIs (3960 ROIs) were reserved for validation during the training process. The remaining 10 breast CT volume data sets (N = 10, ~10%) were used to generate a testing data set. Again, 200 lesion-present ROIs and 200 lesion-absent ROIs were generated from each volume data set resulting in 4000 total testing ROIs. Sample lesion-present ROIs of simulated SKE LKE microcalcifications are shown in Figures 3 & 4.

Figure 3:

Example ROIs of a single simulated microcalcification centered in the field of view. Microcalcification diameter is varied across columns. Native intensity of microcalcifications prior to the blurring procedure ($H$) is varied across rows. Partial volume effects reduce microcalcification conspicuity as microcalcification diameter becomes smaller than the resolution of the imaging system (0.2 mm). Field of view within each ROI: 12 mm × 12 mm.

Figure 4:

Example ROIs of a simulated 0.4 mm microcalcification with 565 HU native intensity displayed using different section thicknesses. Field of view within each ROI: 12 mm × 12 mm.

2.4.2. 3D image generation

3D hybrid image volumes were generated with simulated microcalcification clusters. A single cluster was inserted at the center of every volume of interest (VOI), and microcalcifications of uniform diameters were randomly placed within the cluster. This task had similarities to location-known-exactly tasks because the cluster was centered at a known location, but similarities as well to location-known-statistically tasks because individual calcifications were randomly placed within the cluster across different lesions.

Let $V[i,j,k]$ be a 50 × 50 × 50 anatomical background volume at a random lesion center in the breast. Let $M_n[i,j,k]$ be a volume of zeros of the same size. A spherical boundary is defined within $M_n[i,j,k]$ based on the cluster diameter, $D_{cluster}$, representing the boundary of the cluster. A random number generator was used to define the location, $[i_n,j_n,k_n]$, within the cluster where a blurred microcalcification profile, $m[x,y,z]$, was inserted. $m[x,y,z]$ is inserted at $M_n[i_n,j_n,k_n]$. $M_n[i,j,k]$ is then scaled such that when it is added to $V[i,j,k]$, the peak intensity of $V_{sim}[i,j,k]$ equals $y_n(D_{calc},B,H)$ from Equation 1. Microcalcifications are repeatedly inserted at random locations within the cluster until the desired number of microcalcifications, $N_{calcs}$, are inserted. Because $y_n(D_{calc},B,H)$ is computed at every location $[i_n,j_n,k_n]$ within the cluster, microcalcifications within the same cluster varied in intensity, as is common in breast CT images. Let $[i_{max},j_{max},k_{max}]$ be the indices of the peak intensity voxel in $M_n[i,j,k]$. The simulated VOI with inserted clusters is defined as:

$$V_{sim}[i,j,k]=V[i,j,k]+{\sum }_{n=1}^{N_{calcs}}\left(\frac{y_n(D_{calc},B,H)-V[i_{max},j_{max},k_{max}]}{M_n[i_{max},j_{max},k_{max}]}\times M_n[i,j,k]\right)$$ (3)

$V_{sim}[i,j,k]$ volumes (50 × 50 × 50) were used as lesion-present VOIs, and $V[i,j,k]$ volumes (50 × 50 × 50) generated from separate lesion centers were used as lesion-absent VOIs. Thicker sections were modeled by averaging every $n_{slices}$ slices, resulting in 50 × 50 × N VOIs, where N is defined as:

$$N=\lceil \frac{50}{n_{slices}}\rceil$$ (4)

The maximum intensity projection (MIP) was also generated from each VOI in the axial view plane, resulting in 50 × 50 × 1 ROIs. Five microcalcification diameters $D_{calc}$ (0.20, 0.25, 0.30, 0.35, 0.40 mm), six $N_{calcs}$ (1, 3, 5, 7, 10, 15), and six cluster diameters $D_{cluster}$ (1, 3, 5, 6, 8, 10 mm) were studied. Six section thicknesses (0.2, 0.6, 1.0, 2.2, 3.0, 9.8 mm) in addition to the MIP were studied to understand the role of image display method on microcalcification detectability. The equivalent number of slices $n_{slices}$ for each section thickness was 1, 3, 5, 11, 15, and 49 slices, respectively. Native intensity $H$ was fixed at 565 HU.

Out of the 109 total breast CT volume data sets, 99 data sets (N = 99, ~90%) were used to generate the training data set for training the model observer. For each volume data set, 150 lesion-present VOIs and 150 lesion-absent VOIs were generated from different lesion centers resulting in 29,700 total training VOIs. 10% of the training VOIs (2970 VOIs) were reserved for validation during the training process. The remaining 10 breast CT volume data sets (N = 10, ~10%) were used to generate the testing data set. Again, 150 lesion-present VOIs and 150 lesion-absent VOIs were generated from each volume data set resulting in 3000 total testing VOIs. Sample VOIs of simulated microcalcification clusters displayed using MIP are shown in Figure 5.

Figure 5:

Example images of simulated microcalcification clusters displayed using maximum intensity projection (MIP). Calcification diameter, cluster diameter, and number of calcifications are varied. Field of view within each ROI: 24 mm × 24 mm.

2.5. Detectability estimation using CNN

2.5.1. 2D CNN

A 2D convolutional neural network (CNN) was used to detect simulated microcalcifications in the SKE, LKE image data sets. The input to the CNN was a 2D ROI (120 × 120 × 1), and the output was a decision variable between 0 and 1 scaled by the sigmoid function. The decision variables were used for ROC curve analysis to estimate overall detectability of a data set.

The CNN architecture is shown in Figure 6. The CNN consisted of three layers: two convolutional layers and one fully-connected layer. The three-layered architecture was found to efficiently train the model without leading to overfitting or constraints on memory. The convolutional layers used 3 × 3 convolutional filters with strides of 1. Batch normalization was implemented after the first convolutional layer. Max pooling was then implemented after both convolutional layers with a pool size of 2 × 2 and a stride of 1. Dropout was implemented after both max pooling layers with a rate of 0.2, and after the fully-connected layer with a rate of 0.5. The rectified linear unit (ReLU) activation function was applied in all three layers. The binary-cross entropy (BCE) loss function was used:

$$BCE=-\frac{1}{N}{\sum }_{i=1}^N[y_i\log (\tilde{y})+(1-y_i)\log (1-{\tilde{y}}_{ı})]$$ (5)

where $y$ is the ground truth label (0 or 1), $\tilde{y}$ is the predicted value, and $N$ is the number of samples. Accuracy was one metric used to monitor the model’s performance:

$$Accuracy=\frac{\#correctpredictions}{\#totalpredictions}$$ (6)

while the area under the ROC curve (AUC) was used to evaluate overall detection performance, as described in Section 2.6. The CNN model was implemented in Python using the Keras library³⁵. The Adam optimizer³⁶ was used with a learning rate of 1e-5 and a batch size of 64. Training ran between 50 – 150 epochs. An NVIDIA GeForce GTX 1080 GPU was used.

Figure 6:

2D CNN architecture: two convolutional layers followed by a fully-connected layer.

2.5.2. 3D CNN

A 3D CNN was used to detect simulated microcalcification clusters in the hybrid 3D image VOIs. The input to the CNN was a 50 × 50 × N VOI, where N varied with section thickness, and the output was a scalar-valued decision variable between 0 and 1 scaled by the sigmoid function. The 3D CNN model was designed to mimic the sequential 2D analysis performed by human observers when examining 3D breast CT volumes. Human observers review 3D image volumes slice-by-slice without immediate access to true 3D depth information. Findings from each slice are synthesized into one classification decision for each image volume. To mimic this process, a 3 × 3 × 1 convolutional kernel was used instead of a 3 × 3 × 3 convolutional kernel, which is commonly employed for 3D image data. The 3 × 3 × 1 kernel capitalizes on local spatial information within each slice while disregarding inter-slice information. The CNN synthesizes information from each slice into one decision variable in the fully-connected layer. This kernel choice also suited the range of slices comprising different data sets owing to section thickness: slice averaging across the entire VOI resulted in a single 50 × 50 × 1 slice which required 2D convolutional kernels, while VOIs displayed in the native section thickness contained 50 slices.

The CNN consisted of four layers: three convolutional layers and one fully-connected layer. While the 2D and 3D CNN architectures were similar, an extra convolutional layer was added to the 3D CNN due to the difficulty of detecting relatively small targets (1-15 calcifications) compared to the background space (up to 50³ voxels). Moreover, the microcalcifications in the 3D generated data sets were signal-known-statistically (SKS), whereas the microcalcifications in the 2D generated data sets were SKE, LKE, increasing the complexity of the detection task. All three convolutional layers used 3 × 3 × 1 convolutional kernels with strides of 1. Batch normalization was implemented after the first convolutional layer. Max pooling was implemented after all three convolutional layers with a pool size of 2 × 2 × 1 and a stride of 1. The pooling layers further downscaled the spatial dimensions of the feature maps, reducing computational complexity and providing translational invariance. Dropout was implemented after the three max pooling layers with a rate of 0.2, and after the fully-connected layer with a rate of 0.5 to enhance generalization by randomly deactivating neurons during training. The rectified linear unit (ReLU) activation function was applied in all four layers.

The 3D CNN model was trained using the Adam optimizer with a learning rate of 1e-5, a batch size of 64, and the BCE loss function. Training occurred over 100-300 epochs. The model's performance was evaluated using accuracy and AUC metrics. The CNN was implemented in Python using an NVIDIA GeForce GTX 1080 GPU. The 3D CNN model with the 3 × 3 × 1 convolutional kernel is shown in Figure 7.

Figure 7:

3D CNN architecture: three convolutional layers followed by a fully-connected layer.

2.6. Performance evaluation and statistical analysis

CNN-generated decision variables were used with receiver operating characteristic (ROC) curve analysis to estimate overall detection performance. For all decision variables related to one testing breast CT volume, an empirical ROC curve was constructed by plotting the true positive rate against the false positive rate at various threshold values. The area under the ROC curve (AUC) was computed, signifying detection performance in that breast CT volume. AUCs were computed on a breast-volume basis to elucidate variability related to breast density, anatomical and quantum noise, and motion artifacts. To estimate overall detection performance across all the testing breast CT volumes, the mean and standard deviation of testing-volume AUCs were also computed. In Section 3, $\overline{AUC}$ was plotted with 95% confidence error bars to represent uncertainty in the detectability estimations. 95% confidence error bars were computed using Equation 7:

$${CI}_{95}=\overline{AUC}\pm 1.96\times \frac{\sigma }{\sqrt{K}}$$ (7)

where ${CI}_{95}$ is the 95% confidence interval, $\overline{AUC}$ is the mean AUC across $K$ testing breast CT volumes, $\sigma$ is the standard deviation of AUCs across $K$ testing breast CT volumes, and $K$ is the number of testing volumes ($K=10$). $\overline{AUC}$ was the primary performance metric in this study.

For all comparative tests, the Mann-Whitney U-test was employed. All tests were two-sided, and statistical significance was defined as a difference with a $p$-value less than 0.05. In cases where multiple comparisons were conducted, Bonferroni correction was applied by dividing the desired significance level by the number of comparisons. Statistical analysis was performed using Matlab (Matlab; TheMathWorks Inc., Natick, MA).

3. RESULTS

3.1. SKE, LKE detection task

3.1.1. Effect of microcalcification size and native intensity

SKE LKE detection performance is plotted as a function of microcalcification diameter for three native intensities in Figure 8. When the signal and location are known, microcalcifications that are notably smaller than the resolution of the imaging system (0.20 mm) can be detected despite partial volume effects. Detection performance increases monotonically with calcification diameter and native intensity, and maximum detection performance is achieved when calcification diameter equals or exceeds 0.40 mm. As expected, larger and “brighter” calcifications are more conspicuous than smaller and “dimmer” calcifications. For the remainder of the study, $H=565$ HU is used.

Figure 8:

Effect of size and native intensity on SKE, LKE microcalcification detectability. A single microcalcification was placed at the center of the ROI for this SKE LKE task. AUC is plotted as a function of calcification diameter $D_{calc}$ for three native intensities $H$. Error bars correspond to 95% confidence intervals for each estimate.

3.1.2. Effect of section thickness

SKE LKE detection performance is plotted as a function of section thickness for five calcification diameters in Figure 9. For the largest calcification (0.40 mm), detection performance is generally unaffected as section thickness increases until the thickness reaches 1.0 mm (5 slices), where performance begins to decrease. For calcifications smaller than 0.40 mm, thicker sections reduce detection performance. The native section thickness (0.20 mm) of breast CT enables peak detection performance across all calcification diameters, and detection performance degrades precipitously as section thickness increases.

Figure 9:

Effect of section thickness on SKE LKE microcalcification detectability for five calcification diameters and five section thicknesses (0.2, 0.6, 1.0, 3.0, 11.0 mm). A single microcalcification was placed at the center of the ROI for this SKE LKE task. Error bars correspond to 95% confidence intervals for each estimate.

3.2. Cluster detection task

3.2.1. Effect of image display method

Figure 10 shows the effect of image display method on CNN detection performance of a 5-mm diameter cluster containing 3 microcalcifications. Mean AUC is plotted as a function of section thickness, and detection performance using maximum intensity projection (MIP) is also plotted for five calcification diameters. Data points representing 0.20- and 0.25-mm calcifications across section thicknesses are omitted from the plot for ease in visualization of other differences.

Figure 10:

Effect of image display method. Detection performance using slice averaging (i.e., section thickness) for a 5-mm cluster of 3 calcifications is plotted for three calcification diameters (0.30, 0.35, 0.40 mm) and denoted using solid circles. Detection performance using maximum intensity projection (MIP) is plotted for five calcification diameters (0.20, 0.25, 0.30, 0.35, 0.40 mm) and denoted using solid triangles. Error bars correspond to 95% confidence intervals for each estimate.

Detection performance decreases with increasing section thickness, and peak detection performance occurs using the native section thickness (0.2 mm) and the MIP. When slices are averaged across the entire volume (50 slices or 9.8 mm thickness), microcalcifications of all three sizes are indistinguishable from anatomical background (AUC ~ 0.5). For the three microcalcification sizes, there was no statistically significant difference in detectability between using the MIP and using the native section thickness ($p=.08$, $p=.52$, $p=.76$ for 0.30, 0.35, and 0.40-mm microcalcifications, respectively). The MIP images contained a single slice while the native section thickness images contained 50 slices, thus computing time for MIP images was nearly 50 times shorter than the computing time of the native section thickness images. For the remainder of the study, MIP was used to display the simulated images.

3.2.2. Effect of cluster diameter

Figure 11 shows the effect of cluster diameter on CNN detection performance for clusters containing 3 microcalcifications displayed using MIP. Mean AUC is plotted as a function of cluster diameter for four calcification diameters.

Figure 11:

Effect of cluster diameter on detection performance for clusters of 3 microcalcifications displayed using MIP. Error bars correspond to 95% confidence intervals for each estimate.

The 0.40 mm microcalcification diameter data points frame the diagnostic performance of breast CT when the detection target is 3 calcifications. For the 0.40 mm microcalcification, near optimal detection performance is achieved (AUC ~ 1.0), and cluster diameter has minimal effect on cluster detectability. For microcalcifications on the threshold of detectability (i.e., 0.20 mm, 0.25 mm, and 0.30 mm microcalcifications), cluster diameter becomes important, and larger cluster diameters result in lowered overall detectability, while smaller cluster diameters result in increased overall detectability.

3.2.3. Effect of number of calcifications

Figure 12 displays the effect of the number of calcifications on CNN detection performance for clusters measuring 5-mm in diameter displayed using MIP. Mean AUC is plotted as a function of number of calcifications for four calcification diameters.

Figure 12:

Effect of number of calcifications on detection performance for clusters measuring 5-mm in diameter displayed using MIP. Error bars correspond to 95% confidence intervals for each estimate.

Similar to Figure 11, Figure 12 shows that clusters of relatively large (0.40 mm) microcalcifications are minimally affected by the number of microcalcifications. For microcalcifications at the threshold of detectability (i.e., 0.20 mm, 0.25 mm, and 0.30 mm microcalcifications), the presence of more microcalcifications improves the overall cluster detectability, while fewer calcifications within a cluster decreases the overall cluster detectability.

4. DISCUSSION

Our results demonstrated the utility of the maximum intensity projection (MIP) for displaying image volumes containing microcalcification clusters. We found that there was no statistically significant difference in detection performance when using the MIP compared to the native section thickness, but that thicker sections led to reduced detection performance. This result suggests that the CNN is primarily accessing maximum intensities within breast CT image volumes to determine microcalcification presence. The MIP display method, despite using a single slice, yielded comparable performance to the native section thickness, which employed 50 slices. Reduction in slices did not sacrifice detection accuracy, and this is useful in the context of CNNs, where there are significant computational advantages to using MIPs over multi-slice image volumes. The MIP procedure essentially compresses the 3D image to a 2D image, resulting in efficient and better detection for microcalcifications. Future studies will capitalize on the MIP for investigating additional parameters with improved efficiency. While CNNs do not necessarily predict human observer performance, it is likely that human observers may also benefit from viewing MIPs compared to individual slices.

It is noteworthy that the native section thickness was the optimal section thickness for detecting individual microcalcifications and clusters of microcalcifications. These findings differ from those of previous studies conducted in our laboratory for the detection of mass lesions, where the optimal section thickness for detecting small (1 mm), unenhanced mass lesions was the equivalent of 3-5 reconstructed slices¹⁷. We suspect that the lowered detection performance of mass lesions in the thinnest section was due to the interference of quantum noise with the signal. Microcalcifications are higher-contrast objects compared to mass lesions and may be more immune to quantum noise. One advantage of breast CT over mammography is the ability to adjust the display (section thickness or MIP) of the image volume in real-time using viewing software, and this ability will be important during screening exams when lesion positions are not known a priori, or when lesions of interest vary in morphology.

The effects of cluster diameter and number of calcifications on overall cluster detection performance was also investigated. Our results indicated that cluster diameter affects the detectability of microcalcification clusters, particularly for smaller microcalcifications near the threshold of detectability. Larger cluster diameters resulted in reduced overall detectability, while smaller cluster diameters led to increased detectability. Additionally, we observed that the presence of more calcifications within a cluster improved the overall detectability, while fewer calcifications decreased it. These results begin to elucidate how model observers interact with microcalcification clusters in a 3D volume.

These results underscore the well-known reality that the challenge of detecting microcalcifications is the challenge of resolving small objects. The voxel size of our current breast CT scanner is 0.2 mm. Assuming two calcifications of the same composition but differing only in that one is larger than the voxel size resolution, and the other is smaller than the voxel size resolution, the larger calcification will appear notably brighter when imaged on our breast CT system due to partial volume effects. In this study, we mathematically model the loss of intensity owing to partial volume effects specific to our system, and then evaluate the detectability of partial-volumed calcifications across clinical parameters. These models may be useful to quantitatively estimate the improvement in detectability that may arise from adjusting components of the breast CT system or protocols such as reconstructed voxel size.

This study had a number of limitations. The field of view of the 2D 120 × 120 × 1 ROIs was 24 mm × 24 mm. Due to GPU memory constraints, when generating 3D VOIs, the field of view was reduced to 10 mm × 10 mm × 10 mm (50 × 50 × 50 voxels). The reduced volumetric field of view limited our ability to simulate larger cluster diameters and limited the CNN’s learning of anatomical background. Nevertheless, the extensive training dataset consisting of 29,700 VOIs derived from 109 distinct breast CT datasets likely enabled the CNN to effectively grasp the nuances of breast anatomical background during its training process. Moreover, the methodology for simulating microcalcifications and microcalcification clusters was simplified such that each microcalcification was spherical, and that each cluster contained microcalcifications of homogeneous size. While this simplification does not capture the full complexity of real-world microcalcifications and clusters, it enabled a controlled investigation into specific parameters of interest: microcalcification size, native intensity, cluster diameter, number of calcifications, and image display methods. Contrast imaging was not explored in this study, but initial studies have demonstrated the utility of contrast-enhanced breast CT in improving the conspicuity of small, malignant calcifications such as ductal carcinoma in situ (DCIS)³⁷, and will likely bring added benefit. Future studies should investigate the influence of these additional factors to obtain a more comprehensive understanding of breast CT optimization for microcalcification detection.

5. CONCLUSION

This study investigated individual effects and the interplay of parameters affecting microcalcification detectability in breast CT. We inserted mathematically generated microcalcifications into acquired patient breast CT images and used CNN-based model observers to evaluate microcalcification detectability across clinical and imaging parameters. As breast CT is still a relatively new breast imaging modality, there is an ongoing need to identify optimal imaging protocols. The results of this investigation will be useful for optimizing breast CT imaging protocols for the detection of microcalcifications, a crucial endeavor for the translation of breast CT to the clinic.