Classification of Background Parenchymal Uptake on Molecular Breast Imaging Using a Convolutional Neural Network

Abstract

PURPOSE

Background parenchymal uptake (BPU), which describes the level of radiotracer uptake in normal fibroglandular tissue on molecular breast imaging (MBI), has been identified as a breast cancer risk factor. Our objective was to develop and validate a deep learning model using image convolution to automatically categorize BPU on MBI.

METHODS

MBI examinations obtained for clinical and research purposes from 2004 to 2015 were reviewed to classify the BPU pattern using a standardized five-category scale. Two expert radiologists provided interpretations that were used as the reference standard for modeling. The modeling consisted of training and validating a convolutional neural network to predict BPU. Model performance was summarized in data reserved to test the performance of the algorithm at the per-image and per-breast levels.

RESULTS

Training was performed on 24,639 images from 3,133 unique patients. The model performance on the withheld testing data (6,172 images; 786 patients) was evaluated. Using direct matching on the predicted classification resulted in an accuracy of 69.4% (95% CI, 67.4% to 71.3%), and if prediction within one category was considered, accuracy increased to 96.0% (95% CI, 95.2% to 96.7%). When considering the breast-level prediction of BPU, the accuracy remained strong, with 70.3% (95% CI, 68.0% to 72.6%) and 96.2% (95% CI, 95.3% to 97.2%) for the direct match and allowance for one category, respectively.

CONCLUSION

BPU provided a robust target for training a convolutional neural network. A validated computer algorithm will allow for objective, reproducible encoding of BPU to foster its integration into risk-stratification algorithms.

INTRODUCTION

Molecular breast imaging (MBI) is a nuclear medicine technique that relies on the functional uptake of the radiotracer technetium-99m (^99mTc) sestamibi in metabolically active breast tissue to detect cancer. As a supplemental screening modality, MBI offers increased sensitivity for breast cancer detection, especially among women with dense breasts on mammography.¹ In addition to detecting cancers, MBI also shows variable levels of radiotracer uptake in the normal breast fibroglandular tissue, which has been termed background parenchymal uptake (BPU). A validated lexicon for MBI includes BPU, defined as the relative intensity of radiotracer uptake in fibroglandular tissue relative to fat.^2,3 Recent findings show that BPU on MBI is associated with breast cancer risk, even after mammographic density and hormonal factors are considered.⁴ Because BPU varies among women with similar levels of mammographic density, BPU assessment may offer improved risk prediction or stratification of risk among women with dense breasts.

We hypothesized that BPU on MBI would be amenable to computer-based classification using convolutional neural networks. Such an algorithm would facilitate objective and reproducible classification of BPU on large cohorts, which is needed for risk analyses, and alleviate the need for expensive human assessment of BPU. Therefore, our objective was to develop and validate a deep learning model using image convolution to automatically categorize BPU on MBI.

METHODS

Study Population

This retrospective analysis was compliant with the US Health Insurance Portability and Accountability Act and approved by the Mayo Clinic Institutional Review Board, which issued a waiver of informed consent. MBI examinations for either clinical indications or research purposes were annotated for research purposes and stored in a research registry. Briefly, an MBI examination uses an intravenous injection of ^99mTc sestamibi to localize abnormal findings in the breast.⁵ Each MBI examination is obtained using a dedicated dual-head gamma camera system equipped with semiconductor-based detectors. The modality provides two-view acquisitions (craniocaudal [CC] and mediolateral oblique [MLO] analogous positioning) of each breast using both an upper and lower detector simultaneously. Additional breast views may be acquired as needed but were not included in our analysis. Therefore, each image data set consisted of eight images: CC and MLO projections of left and right breasts acquired with two detector heads of the dual-head system. BPU was an interpretation stored in this registry and the subject of this examination. Minimal exclusion criteria were used to remove cases where sestamibi was used primarily for cardiac imaging purposes and where images were denoted as poor quality by the radiologist during review.

BPU Lexicon and Establishment of the Reference Standard

The established BPU lexicon consists of four categories: photopenic, minimal-mild, moderate, or marked²; however, the registry was evaluated according to a five-category variation obtained by splitting the minimal-mild classification into two separate levels. BPU describes the relative intensity of radiotracer uptake in fibroglandular tissue compared with that in subcutaneous fat. Representative images for this five-category system are presented as Figure 1. Two expert radiologists with more than 5 years of experience in evaluating MBI examinations graded BPU according to the lexicon. Radiologists had access to recent mammograms but were blinded to all other clinical data at time of review. A specialized workstation was created to view, including windowing and leveling controls, and annotate the interpretation of the image set. Given the large volume of MBIs to read, BPU was evaluated with a single reading by one radiologist. A set of 10 images, including two representing each lexicon category, were presented repeatedly throughout the reading process to understand variation of BPU classification within reader and between readers over time.

FIG 1.

Example of molecular breast images representing the range of uptake: (A) photopenic, (B) minimal, (C) mild, (D) moderate, and (E) marked.

Image Preprocessing

The analysis started using the raw digital imaging and communications in medicine (DICOM) files without prespecified windowing or leveling values. Although the native resolution of the collectors was 128 × 96 pixels (20 × 15 cm field of view), the acquired image was stored as a 128 × 128 pixel array in the DICOM files. The pixel array values, which represent the intensity of the radiotracer uptake in the voxel, were normalized to be between 0 and 1, to represent a gray-scale gradient. To avoid the need for manual brightness adjustment and to account for a small percentage of pixel array values > 1 after normalization (ie, potential corruption), all images were processed using a three-part filtering approach. First, any normalized pixel value that exceeded 1 was winsorized to a value of 1. Next, the pixel values were further restricted to be no greater than the 99.99th percentile (ie, second-brightest pixel). Finally, any values < 0.01 were set to 0.01 to provide a more uniform black in the image. Some images included radioactive markers external to the breast to indicate left and right breast (a detector-visible R or L region outside the breast region). These targets were left in the image with the intention to evaluate how well the algorithm could be trained using images with minimal preprocessing.

Model Architecture and Hyperparameter Optimization

At a high level, the deep learning model consisted of a combination of convolution/pooling steps followed by dense fully connected layers. Data were partitioned into two mutually exclusive data sets: training (80%) and testing (20%). Internal validation of data (random subsplitting of the training data) was performed to evaluate hyperparameter optimization. As part of the model optimization process, the standard tunable hyperparameters were allowed to vary using a defined set of values. These hyperparameters included the number of convolutional filters, filter sizes, mean versus max pooling, activation functions, number of dense layers, and number of hidden nodes per dense layer, among many other parameters, to control early termination and internal validation strategies. Hyperparameter searching was facilitated using the tfruns package available in R software (R Foundation, Vienna, Austria). This package recorded all details pertaining to model parameters and model performance to facilitate subsequent analyses. An example of training environment and use of tfruns has been made publically available to help support transparency and reproducibility in research.⁶

Although each breast contained up to four images and most women had two breasts studied with MBI, each of the individual eight images was used independently of the patient. However, because BPU was a breast-level determination, each of the four images per breast was assigned the same BPU determination for training purposes. Two training targets were considered in the development of the final algorithm. The first approach dichotomized BPU into photopenic and minimal uptake versus mild, moderate, and marked uptake to leverage what was considered an easier (binary) target less sensitive to case imbalance across the five categories. The binary model was used to evaluate various architectures and narrow the range for hyperparameters for the desired training target (the five-category BPU classification). Once a collection of model architecture parameters was known to provide training success, training of the final model commenced.

Upon selection of the final model architecture, the training program was modified to allow for adjustment to the learning rate on learning plateau and to provide a larger number of possible training cycles (epochs). The final model architecture was saved to provide a means of formally evaluating performance. The architecture of the model is shown in Appendix Figure A1.

Statistical Considerations

To assess model performance, standard measures of diagnostic model performance were used: area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. CIs for AUC were based on Sun and Xu’s optimization of DeLong’s algorithm⁷ as implemented in the pROC R package (version 1.21.1).⁸ For sensitivity and specificity, values were tabulated for interpretation but were not summarized because of the clustering of images within breast and patient. For the primary figure of merit (classification accuracy), generalized estimating equations were used to provide the CI for measures of accuracy while accounting for the clustering of the results within patient (ie, up to eight images per person) or, in some analyses, clustering of predictions within breast. Two measures of accuracy were considered. The first was the standard measure where the predicted BPU needed to match the reference BPU for each image. A second measure of accuracy allowed for a one-category difference in predicted BPU. In addition to these measures and consistent with prior reports using BPU, binary categorizations were also derived by grouping BPU into photopenic and minimal versus not and photopenic, minimal, and mild versus moderate and marked. The probability of the groups was derived by summing the probabilities obtained from the five-category model. In addition, model performance at the breast level was also evaluated to mirror radiologist interpretation. Specifically, a summarized measure of performance was also calculated by taking the majority classification across the four images per breast, allowing for both direct matching and one-category off predictions to determine if the algorithm correctly classified BPU at the breast level.

The convolutional neural network was estimated using Keras TensorFlow 1.5 complied from source code to enable GPU support (NVIDIA Titan Xp) on a Macintosh running High Sierra (version 10.13.5). Calculations were supported by Python (version 3.6.4; Python Software Foundation, Wilmington, DE), NumPy (version 1.14.0), Keras (version 2.2.0), and CUDA (version 9.1). Grid optimization for the hyperparameters was performed using the tfruns package through the R interface to TensorFlow⁹ using R (version 3.5.0). To facilitate reproducible research, the final model architecture is summarized in Appendix Figure A1, and a video recording of the computing environment and programming code has also been made available.⁶ The raw imaging files will not be made available.

RESULTS

Image Inclusion Summary

The registry of all MBI examinations contained 6,062 examinations consisting of 5,351 unique patients (Fig 2). After exclusions, a total of 3,919 examinations were randomly divided into development (training) and external validation (test) data sets. This partitioning was constructed to ensure no patient was included in both the training and test data sets. Because each patient may have had up to eight images per examination (two views, MLO and CC; two gamma cameras, upper and lower), the total number of images for analysis was 30,811, which represents approximately 7.9 images for participant. There were 541 images excluded for having the wrong spatial resolution (array size different from 128 × 128 pixels) or corruption in the DICOM files rendering them unreadable during data importation. The frequency distribution of BPU along with the distribution of the limited clinical attributes available to this study is listed in Appendix Table A1.

FIG 2.

Image and participant selection. BPU, background parenchymal uptake; DICOM, digital imaging and communications in medicine.

Single-Image Classification Performance

Table 1 summarizes the model performance on the external test data. Accuracy of 69.4% (95% CI, 67.4% to 71.3%) was obtained using the direct match criterion. When the prediction was allowed to vary ± one category (light green shaded region), the accuracy increased to 96.0% (95% CI, 95.2% to 96.7%). This increase in accuracy is illustrated by the examination of predicted class probabilities (Fig 3). As shown in the figure, when an image was categorized as photopenic by the reference reader, the algorithm provided high probabilities for either photopenic or minimal. Similarly, when an image was considered as having marked uptake, the algorithm provided relatively high probabilities that the image was either moderate or marked. This lack of discrimination between other adjacent categories lowered the overall accuracy. However, prior research has used categorized BPU for this reason.^1,4,10,11 Although a model was trained using a dichotomized BPU classification in the early stages of the research, the five predicted probabilities could be summed to provide any of a number of combinations of categories without the need for extensive model retraining.

TABLE 1.

Confusion Matrix of Model Performance on External Testing Data

FIG 3.

Distribution of predicted probabilities for each of five classifications of background parenchymal uptake (BPU): (A) photopenic, (B) minimal, (C) mild, (D) moderate, and (E) marked. Stratified by true BPU status, as indicated in the shaded region at the top of each panel. The model produces a predicted probability for each of the five possible classes; these estimates are shown as box and whisker plots. The figure illustrates how the classes with the highest predicted probabilities are qualitatively aligned with true BPU category.

Two categorizations were selected for this research (Fig 4). The first categorization combined photopenic and minimal together as a classification representing subtle uptake patterns. This resulted in an accuracy of 85.3% (95% CI, 83.8% to 86.9%) and an AUC of 90.4% (95% CI, 89.5% to 91.2%; Tables 2 and 3). Model performance increased further when the mild category was included in the subtle uptake group. In particular, accuracy and AUC numerically increased to 93.0% (95% CI, 92.0% to 94.0%) and 95.3% (95% CI, 94.6% to 96.0%), respectively.

FIG 4.

Performance of the model at predicting categorized background parenchymal uptake (BPU). (A) Mild uptake with moderate and marked uptake, and (B) receiver operating characteristic curve (ROC). (C) Mild update with photopenic and minimal uptake, and (D) ROC. Each panel includes (A, C) a box plot of predicted probabilities for having the less pronounced uptake classification along with (B, D) the associated ROC curve.

TABLE 2.

Confusion Matrices for Categorized BPU Prediction

TABLE 3.

Summary Measures of Performance

Breast-Level Prediction

A natural limitation of the image-level interpretation is that the clinical determination of BPU was based on the set of four images per breast, whereas model training and individual predictions were at the image level. To evaluate the performance of the model comparable to interpretation provided by the radiologist, a breast-level classification was derived from the individual image predictions. With the five-category lexicon and a breast-level prediction of BPU, the accuracy remained strong, at 70.3% (CI, 68.0% to 72.6%) and 96.2% (CI, 95.3% to 97.2%) for direct match and allowance for one category, respectively.

Intrareader and Interreader Reproducibility

Appendix Figure A2 shows the results of repeated presentations of the same 10 cases to the two study readers over the entire study reading period. Reader 1 classified five of the 10 cases identically over the study, and of the remaining five cases, four of these had fewer than four discordant ratings. However, reader 2 provided fewer concordant interpretations over the course of the study. Specifically, this reader provided consistent interpretations for four cases. The remaining six cases demonstrated one-category fluctuations in the readings. Cases 4, 5, 9, and 10 showed some indication for variation in scale usage between readers; however, across all 529 replicated readings over all 10 cases, all but one reading was within ± one BPU category of the participant mode.

Assessment of Model Optimism

All results presented thus far are based on external test data that were not used to train the model. Attenuation of the performance metrics from the training data to the test data can be used to assess model optimism. For this study, the percentage of change (training value) –was 4.9% (72.9%), −1.1% (96.4%), −4.2% (73.4%), and −0.5% (96.7%) for image-based accuracy, image-based one-category accuracy, per-breast accuracy, and per-breast one-category accuracy, respectively.

DISCUSSION

Objective, reproducible classification of BPU is considered an essential component of understanding how BPU may be related to breast cancer risk. This research developed and validated an automatic algorithm for classification of BPU using the standardized lexicon for BPU. Model performance was found to be high and robust to modeling assumptions. There was reader variation in the radiologists’ interpretations over the standard set of 10 control images, which suggests that objective, reproducible, computer-based classification could be useful for standardizing risk prediction.

All breast imaging modalities have lexicon terms to describe the appearance of normal fibroglandular tissue, such as density on mammography, background parenchymal enhancement on magnetic resonance imaging (MRI), and BPU on MBI. Although background appearance is important in stratifying patients and may affect which screening and preventive options might be offered to a patient, its usefulness is hindered by significant inter- and intraobserver variability.¹² For example, patients may be offered supplemental screening based on having dense breasts (ie, Breast Imaging Reporting and Data System category c or d), but because of reader variability, they may intermittently be assigned as having c density some years and b density other years. If the patient lives in a state requiring density notification, she would some years get a letter recommending that she consider supplemental screening, and other years, her letter would tell her she does not need it.

Furthermore, functional imaging findings (background parenchymal enhancement obtained from an MRI examination and BPU obtained during an MBI examination) have been shown to change in response to treatment or physiologic changes in the patient; for example, background parenchymal enhancement on MRI has been shown to decrease in intensity when patients start receiving tamoxifen or aromatase inhibitors.^13-15 Anecdotally, we have observed the same phenomenon for BPU on MBI. Investigation is necessary to determine if changes in BPU over time reflect changes in risk and whether changes in BPU could predict the efficacy of chemopreventive treatment. Again, the value of such a test would be significant but is hindered by variability in assessment by human readers and our insensitivity to detect and classify subtle small changes in uptake or enhancement. The consistency and sensitivity of an automated system could aid our ability to understand patients’ risk for breast cancer and how to modify it.

The BPU lexicon has been developed to mimic the background parenchymal enhancement classification on breast MRI (minimal, mild, moderate, marked) with the addition of a fifth category of photopenic, which is unique to nuclear medicine imaging. However, there are some questions about how discernable subtle nuances in lexicon terms can be used to differentiate classes. An advantage of an algorithm is that this subtle distinction is not needed, because the computer can assign a classification independent of words used to describe the grades of uptake in the MBI images.

Future research is needed to expand the clinical decision support tools beyond automated classification of BPU obtained in this study. One goal of the registry supporting this research is to better understand how marked BPU differentiates from high radiotracer concentrations associated with a tumor. As the registry grows and follow-up data on incident cancers become available, we will strive to develop models for this purpose. Such models may help inform patient management by providing additional actionable information about an examination. Future research will also seek to examine the relationship of breast cancer risk with more generalized tracer uptake patterns, whether classified according to the standardized lexicon or perhaps a novel machine-based quantification of tracer patterns. Ultimately, we do not believe that breast cancer risk will be a function of tracer uptake pattern alone, so modeling will therefore need to consider the integration of traditional risk factors.

A limitation of this research is that the cohort did not have BPU recorded by multiple radiologists. Furthermore, we did not conduct a human performance study with which we could formally compare the algorithm. Future research will be needed to determine if the model developed here is more, or possibly less, reliable and accurate than practicing radiologists. Along these same lines, we anticipate that some of our reference standard ratings may be imperfect and would be reported differently if the image were reread based on our intra- and interreader reproducibility study. We attempted to address this by examining accuracy, allowing for a single-category difference in rating, a threshold of difference that appears to be well aligned with how the expert radiologist used the scale over the duration of the study. Some additional challenges of using convolutional neural networks for computer vision tasks such as ours are worth noting. First, these networks have introduced a nomenclature to describe the model architecture that may not be familiar to all researchers. Terms such as hyperparameters, filters, and fully connected layers need to be understood to fully appreciate the nuances of model building. Fortunately, there have been excellent primers made available recently.^16,17 Such resources help translate the advanced mathematic concepts into analogies and visual representations of the key steps in the modeling process. A second challenge worth noting is that the convolutional neural networks by themselves may only solve singular steps in the overarching analysis. In the case of our study, only the MBI image was used in the classification of BPU. More advanced approaches are needed to combine the images with other relevant clinical data to refine predictions. Finally, the training is inherently empiric, and the final model architecture is a balance of complexity, performance, and our current understanding of state-of-the-art approaches. We feel we have found a final model that seems robust to new data, but we also know that advances in the computer vision field will enable us to improve upon the model in time.

In conclusion, we successfully developed and validated a convolutional neural network for the automatic classification of BPU in women undergoing supplemental screening using MBI. Additional research to determine if these automatic classifications add to risk stratification and breast cancer risk prediction is warranted.

Appendix

FIG A1.

Final network architecture for predicting five-category background parenchymal uptake (BPU) classification. Model is conceptually broken into two key elements: convolutional cycles and deep fully connected layers. Convolutional cycles contain repeated convolutions that apply varying numbers of filters (essentially a machine-learned pattern in the image) to successively extract additional data from model input (a molecular breast image, in this case). Pooling of data is used to reduce dimensionality of input by averaging pixels in a local region. This process generates new representations of input to support additional imaging feature extraction during model training. For both convolution filters and pooling steps, the local region in terms of pixel dimensions is denoted in parentheses (eg, 3 × 3 grids were used for first convolutional set). Normalization and activation functions are included in model to help mitigate overtraining. Rectified linear unit (Relu) activation constrains normalized values < 0 to be 0 and allows values > 0 to remain unconstrained. Leaky Relu instead greatly shrinks (moves toward 0) negative normalized values, thereby allowing those inputs to assist with classification. For brevity of presentation, blocks of model architecture that are repeated multiple times are indicated as multiples in the figure. Dense fully connected layers represent the deep learning part of the model. The first step in this process is to restructure data from square image format to traditional single-dimension numeric vector with the flatten command. Number of layers and number of nodes in hidden layers are indicated in the figure. Model concludes with a final activation step that produces predicted probability for each of the five BPU classifications.

FIG A2.

Reader variation over a standard set of 10 patient examinations randomly inserted into reading sessions to assess consistency of coding over the entire study duration.

TABLE A1.

Summary of Available Data in Training and Testing Data Sets

AUTHOR CONTRIBUTIONS

Conception and design: Rickey E. Carter, Jennifer R. Geske, Dana H. Whaley, Deborah J. Rhodes, Carrie B. Hruska

Financial support: Rickey E. Carter, Carrie B. Hruska

Provision of study material or patients: Carrie B. Hruska

Collection and assembly of data: Jennifer R. Geske, Dana H. Whaley, Katie N. Hunt, Michael K. O’Connor, Carrie B. Hruska

Data analysis and interpretation: Rickey E. Carter, Zachi I. Attia, Jennifer R. Geske, Amy Lynn Conners, Dana H. Whaley, Katie N. Hunt, Carrie B. Hruska

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/site/ifc.

Zachi I. Attia

Patents, Royalties, Other Intellectual Property: Licensed technology to AliveCor (Mountainview, CA) but have not received any royalties to date (Inst)

Michael O’Connor

Patents, Royalties, Other Intellectual Property: Patents on the MBI technology soon to be licensed to CMR Naviscan (Inst)

Carrie B. Hruska

Patents, Royalties, Other Intellectual Property: Royalties for MBI technologies licensed to CMR Naviscan; licensing agreement with CMR Naviscan (Inst)

No other potential conflicts of interest were reported.