Digital Analysis in Breast Imaging

Abstract

Breast imaging is a multimodal approach that plays an essential role in the diagnosis of breast cancer. Mammography, sonography, magnetic resonance, and image-guided biopsy are imaging techniques used to search for malignant changes in the breast or precursors of malignant changes in, e.g., screening programs or follow-ups after breast cancer treatment. However, these methods still have some disadvantages such as interobserver variability and the mammography sensitivity in women with radiologically dense breasts. In order to overcome these difficulties and decrease the number of false positive findings, improvements in imaging analysis with the help of artificial intelligence are constantly being developed and tested. In addition, the extraction and correlation of imaging features with special tumor characteristics and genetics of the patients in order to get more information about treatment response, prognosis, and also cancer risk are coming more and more in focus. The aim of this review is to address recent developments in digital analysis of images and demonstrate their potential value in multimodal breast imaging.

Introduction

In Europe, breast cancer (BC) is the leading cause of death from cancer in women, and it is the most commonly diagnosed cancer worldwide except in Eastern Africa. The prevalence of BC has been increasing due to the possibility of early BC diagnosis, population aging, and changes in risk factors, but independent of further development of new treatment options, early diagnosis is very important as it increases the chances of survival [1].

Breast imaging is a subspecialty in the diagnostic field of radiology, which involves several imaging techniques for the early detection of BC. It also plays a fundamental role in the assessment and follow-up of BC [2, 3, 4, 5]. It requires a multimodal approach that includes mammography (MG) as a gold standard supplemented by sonography (US) and dynamic contrast-enhanced magnetic resonance tomography (DCE-MRI). These techniques are used to search for malignant changes in the breast, e.g., in the screening program, intensified screening, or follow-up monitoring [6, 7].

Although breast imaging is currently the most effective tool for detecting BC, there are still deficits in the detection and interpretation of lesions by the human reader [8]. In addition, the number of dedicated breast imager is limited. For this reason, improvements in breast imaging and image-based features are crucial and currently a very active topic in radiological research. Image analysis with the help of computers has shown that computerized feature extraction and new algorithms classification are great tools to assist physicians in the detection and classification of abnormalities [9]. Moreover, they are playing an increasingly important role in predicting clinical prognosis [10].

This review addresses recent developments in digital analysis and their potential value in the context of multimodal breast imaging.

Mammography

A mammogram is an X-ray method for breast imaging that has been constantly evolving in recent years from a film-based analog MG in the 1960s to full-field digital techniques. While analog MG can lead to an inaccurate diagnosis or false positive results, digital MG is more robust and superior, especially in lesion detection in dense breasts or premenopausal woman [11]. In addition, the radiation exposure of digital MG is around 20–30% lower than screen film for MG [12]. Furthermore, the images can be subsequently altered and easily exchanged electronically.

Since to detect an ion of masses is usually not the biggest challenge for radiologists, if they are not obscured by overlying dense breast tissue, the interpretation is trickier. On the other hand, the detection of small microcalc clusters after a long day or reading a big batch of images in screening can become difficult due to lack of concentration. In order to support the radiologists' detection rate and interpretation of breast imaging, innovative image-based features such as computer-assisted diagnostic (CAD) systems have been developed. CAD systems identify abnormalities on the breast image and can be used after or before the radiological interpretation of the image [13, 14, 15, 16]. Even though these systems can be applied on digitized analog images, the development of digital MG made it much easier to apply these computer tools and will also enable artificial intelligence algorithm in the future.

Magnetic Resonance-Mammography

MRI [17] is usually used as a complement to X-ray MG for the diagnosis of BC. It offers the advantage of providing not only 3-dimensional spatial information but also temporal information. DCE-MRI in breast imaging is considered to be the technique with the highest sensitivity, exceeding 90% [18]. Current data also confirm high accuracies in the detection of ductal carcinomas in situ or the diagnosis of lesions presenting with microcalcifications [19, 20, 21, 22].

MRI allows the acquisition of morphological and functional information of breast tissue and breast lesions and the evaluation of pathological features. Functional information can be obtained using two different techniques: dynamic and morphological enhancement assessment [23] and diffusion-weighted imaging (DWI) [24, 25]. The contrast dynamic technique is based on different enhancement patterns of breast lesions over time after the administration of a contrast agent [26, 27, 28]. DWI, on the other hand, is a technique that measures the diffusion of water molecules in tissue and does not require a contrast medium. Due to the water molecules' mobility, DWI reflects biophysical properties such as cell density, integrity of the membrane, and tissue microstructure [25, 29]. MRI, however, is a method that requires a high level of experience, which may be limited [30, 31]. Besides, image interpretation can show inter- and intraobserver variations [32]. Therefore, CAD programs were also developed for MRI in order to detect and clarify suspicious image features, to assist radiologists in distinguishing lesions, and to save time [33, 34, 35, 36].

Computer Assistance for Image-Based Diagnosis of BC

The long history of CAD systems began between 1960 and 1970, when research studies presented a new automatic computer model that was able to detect and classify suspicious abnormalities [37]. It the 1980s, medical physicists and physicians at the Kurt Rossmann Laboratories (Department of Radiology at the University of Chicago) developed two comprehensive categories of computer models: computer-aided diagnosis (CADx) and computer-assisted detection (CADe) [38].

CAD systems analyze the mammographic images and mark abnormal densities and microcalcifications assisting the radiologist to incorporate the machine results into their decision-making process. The initial systems had a high rate of false positive marks, and not every suspicious finding was safely marked [39]. Therefore, CAD systems should only be used as support for radiologists; the responsibility lies solely with the physician. CAD systems are nowadays widely accepted and used throughout the field of radiology [16, 40]. The image analysis of CAD systems is based on image acquisition, segmentation, processing, detection, and output [15, 16, 41]. CAD can be applied in screen-film MG as well as digital MG. However, conventional images must be digitized first, as CAD systems can only analyze digital datasets [42].

Emerging studies introduce new CAD systems based on neural networks. These innovative systems have the supplementary ability to use already collected information as a pool of data in order to compare with new mammograms [43]. With the new technology, researchers hope that the new CAD method will replace the second imaging reader in the future [44, 45].

The CAD system in MG is used as an assistant to the radiologist by enhancing potential findings that may get overlooked and by reducing the time of image interpretation. CAD in MG is advantageous when there is a high inconsistency among observers, a lack of trained observers, or double reading with two or more radiologists is impractical [46]. Clinical studies have shown that the addition of CAD leads to a sensitivity increase from 20 to 21% in the detection of BC by radiologists [46, 47, 48, 49].

CAD for breast MRI has a different function from its application in MG. It is a component in the daily routine that serves more like a computer-assisted evaluation and visualization tool than only as a lesion detection [33]. Radiologists usually detect the lesions easily, but only the most experienced ones can safely assess the lesion [50]. MRI interpretation requires knowledge of the morphology and kinetics of the lesion as well as its enhancement patterns. Malignant tumors tend to enhance and wash out quickly. An abnormal suspicious morphology of a lesion added to type III kinetics [34] indicates malignancy and increases the likelihood of biopsy.

CAD is a system able to organize this multifaceted image information, reducing the number of false positives and, thereby, unnecessary biopsies [51]. In addition, it has been shown that CAD may also be useful for assessing changes in enhancement patterns of tumors after chemotherapy [52].

Breast Density Assessment

In order to be able to predict the individual risk of women to develop BC, researchers' interest in imaging risk indicators, such as breast density, fibroglandular tissue (FGT), and background parenchymal enhancement (BPE) after approximately 90 s, has increased in recent years. The assessment of the density of the breast using the BI-RADS lexicon quite often shows a significant variability among readers. Therefore, more objective measurement tools would be helpful. Automatic assessment of volumetric percentage density using dedicated software reduces these variabilities. Studies have shown that mammographic percent density, BPE, and FGT are independent imaging biomarkers as they are also risk factors for the development of BC [53, 54, 55].

In addition, several research studies assessed whether a computer-assisted program would also find less BC due to increased tissue density of the breast: Bolivar et al. [56] presented a significant difference in sensitivity between the nondense breasts compared to dense breasts (95 vs. 89%) after using a CAD system. In contrast, Brem et al. [47], Birdwell et al. [57], and Yang et al. [58] showed no impact of breast density on the performance of CAD systems. Engelken et al. [39] showed an influence of the volume of breast tissue but not its percentage on the performance of a CAD system. However, the groups used different CAD systems and partly digital partly screen-film mammograms.

Breast density in MRI is a volumetric assessment of FGT, since fat and FGT are two major components of breast MRI. It is strongly correlated with mammographic percent density [59, 60] and associated with BC risk [55]. BPE in MRI is recognized as enhancement of normal breast tissue to varying degrees after injection of the contrast agent [61, 62]. Breast density, FGT, and BPE may be altered as they represent dynamic physiological processes subject to hormonal changes [63]. In order to assess these imaging biomarkers, radiologists use a 4-point scale that qualifies the breast density according to a visual criterion established by the American College of Radiology: Breast Imaging Reporting and Data System (BI-RADS) criteria. In MRI, BI-RADS assessment should occur on the first image after contrast injection at approximately 90 s. However, the manual assessment of BI-RADS can lead to inconsistencies between inter- and intrareaders and obstructs the prediction of BC [64, 65]. To overcome these limitations, other algorithms are being developed [66, 67].

The initial computer-assisted systems and software algorithms where using more basic techniques and were limited by the performance speed of the available computer technique at that time. But in the last years, more and more sophisticated computer techniques using neuronal networks, deep learning (DL), and artificial intelligence came more and more into focus. These techniques also enable researchers to extract multiple imaging features and correlate them to biological markers to predict, for example, the type and behavior of a tumor. These techniques are described in more detail in the following section.

Radiomics and DL

Radiomics is a technique that converts radiological imaging data, such as CT or MR images, into a high-dimensional feature space, where the term feature refers to a scalar number that encodes a specific property of the tumor. This feature space is hypothesized to contain a link to a clinical or biological endpoint – such as tumor (sub-)type, prognosis, or treatment response. Computational modelling and machine learning (ML) approaches are often used in a hypothesis-generating approach to link the high-dimensional image feature space to a specific endpoint. As such, radiomics can be seen as the radiological interpretation of the systems biology approach [68]: Systems biology is the study of complex biologic systems, which relies on integrating multifaceted data about genes, metabolism, protein expression, and other components. Systems biology approaches encompass data from many scientific disciplines and form an integral part of precision medicine.

Technical Introduction

Radiomics was introduced in a ground-breaking work by Aerts et al. [69]. This work demonstrated in independent CT datasets of patients with lung or head-and-neck cancer that radiomics features, generated by automated image analysis, contain prognostic information. Fundamentally, this work demonstrated that imaging phenotype becomes accessible to computational analysis by the proposed approach termed “radiomics,” which is described in the following subsections.

This initial paper was followed by an abundance of publications which investigated radiomics approaches for the analysis of CT and MR data, with the clinical endpoints of classification, e.g., of tumor types or the presence or absence of genetic subtypes, or progression-free or overall survival [70]. It is noteworthy that the majority of radiomics studies has focused on oncologic questions, although, in principle, the radiomics approach is feasible for all questions that can be formulated as classification, regression, or survival analysis. For the sake of simplicity, the following discussion is limited to radiomics analysis of tumors.

Figure 1 outlines a typical radiomics analysis. Starting with high-quality tomographic images, the tumor is segmented or contoured on each image slice, resulting in a 3-dimensional definition of the tumor shape. In the initial radiomics studies, this tumor segmentation was performed manually by expert readers, typically radiologists with sufficient domain expertise. Depending on image contrast, tumor boundaries, and image resolution, this manual segmentation can be an extremely time-consuming and user-dependent task.

Fig. 1.

Outline of a state-of-the-art radiomics analysis [88]. Briefly, a tumor is outlined or contoured on tomographic images. In the next step, a range of quantitative image features is computed, typically comprising shape features, signal intensity distributions, and image texture. Finally, machine learning is used to establish a link between this high-dimensional data space and the relevant clinical endpoint or outcome.

Once the tumor is segmented, radiomics analysis proceeds by calculating a large number of quantitative image features, which can roughly be categorized into three groups. The first group of features describes tumor shape and encompasses simple and well-understood parameters such as maximum diameter, tumor volume, or tumor surface area. Often, additional parameters such as sphericity, the ratio of surface to volume, or other parameters are calculated. The second group of image features describes the histogram of signal intensities within the tumor. Again, this encompasses simple and straightforward parameters such as mean or median, minimum or maximum signal intensity, but also energy, skewness, and kurtosis of the histogram. With these groups of features, the tumor is already well characterized, but the relation of neighboring pixels is not taken into account. To this end, the third group of features comprises image texture, which essentially captures the spatial relations of different gray levels, by means of gray-level co-occurrence or gray-level run-length matrices. Notably, these texture features are able to capture various aspects of tumor heterogeneity, a measure which often has strong prognostic relevance.

After feature extraction, imaging data are converted into a long vector of image features. These data can be readily enhanced with additional information, such as demographic, clinical, or pathological data, or even additional -omics data, e.g., from genomics. This wealth of data, however, should be interpreted with caution: it may well be that a large number of these features is of little use for endpoint prediction. For example, it may be that some image features are highly sensitive with respect to image segmentation, so that feature changes are dominated by small changes in segmentation, as, for example, caused by interreader variability. Other features may have no association with the outcome at all, and again others may have high intercorrelation and, thus, act as confounders in further analysis. For this reason, the high-dimensional feature space is often narrowed down by means of feature selection, which may employ a large number of feature quality measures to narrow down the number of predictors.

The link between feature space and clinical endpoint then needs to be provided either by statistical modeling or ML, often in a classification or survival setting. In ML terms, this problem can be formulated as “supervised learning:” for each patient, the feature vector and the associated label, i.e., the clinical endpoint, is known; the ML model needs to establish the link between the two entities, which is performed in a so-called training step. After training, the model can then predict an outcome for new hitherto unseen patient data. During training, special care needs to be taken in order to arrive at meaningful predictions – otherwise, a model might simply memorize training data and be unable to predict on new data. A popular strategy for this purpose is to split the data into training and testing data; during training, the model sees only the training data, and after training, the model performance is evaluated on the testing data. A generalization of this approach is cross validation: here, the data is divided into several folds, one fold is held out during training and used only for prediction. This is then permuted, so that predictions for each fold are based on training with all other folds.

Applications in BC

The key application of radiomics in oncological settings is the prediction of a clinical endpoint. In the context of BC, radiomics applications have up to now focused on the characterization of suspicious lesions, on the characterization of tumors, and on prognosis and risk prediction. In the following, we present key studies in these fields. For a more comprehensive overview, please refer, for example, to Pinker et al. [68] or Crivelli et al. [71]. It should be noted, however, that most of these studies have been performed in a retrospective fashion and require validation in prospective studies.

The diagnostic workup of suspicious lesions on conventional MG is a central application of MRI in BC. The additional value of radiomics feature as compared to the maximum linear size was evaluated by Whitney et al. [72], their study demonstrated that radiomics features improved the ability to distinguish benign lesions from luminal A BCs. A distinguishing feature of MRI is its ability to provide information beyond pure morphology, a very useful application is imaging of water diffusion in tissue, based on the notion that densely packed tumor cells provide more diffusion restriction than healthy tissue and, thus, appear dark in so-called apparent diffusion coefficient images. Radiomics analyses of diffusion-weighted MRI in BC have been shown to be superior to mere mean apparent diffusion coefficient values in the discrimination between benign and malignant lesions [73].

In a prospective bi-centric study [74], a specialized DWI protocol was used for imaging patients with suspicious lesions detected on X-ray MG. Here, the radiomics analysis was demonstrated to have higher specificity than conventional image analysis, even with MR scanners from different vendors. Radiomics has also been used to differentiate BC molecular subtypes. Here, it was demonstrated that a quantitative MRI radiomics approach may be very helpful for the molecular classification of invasive BC – e.g., the differentiation of ER+ versus ER– or HER2+ versus HER2– receptor status [75]. In a similar approach, radiomics analysis of DCE-MRI was used to differentiate luminal A, luminal B, HER2, and basal-like subtypes [76].

With respect to prognosis, Li et al. [77] aimed to use MRI features to predict BC recurrence scores from multigene assays. The findings of their study suggested that radiomics features used for image-based phenotyping may be helpful in assessing the risk of cancer recurrence. With respect to the endpoint of disease-free survival, Park et al. [78] demonstrated that a radiomics signature from preoperative MRI, termed “Rad-Score,” has prognostic value. In particular, when combined with clinicopathological and MRI findings, this signature improved the estimation of disease-free survival, suggesting that the radiomics signature provides additional prognostic value. Finally, a recent study investigated the potential of radiomics to predict pathological complete response of patients with locally advanced BC under neoadjuvant chemotherapy [79, 80].

Deep Learning

In contrast to radiomics, where image features are “handcrafted,” DL approaches promise to learn relevant image features on their own. By combining image analysis and ML in one network, convolutional neural networks (CNN) have had a disruptive effect on a large number of ML approaches, in particular with respect to computer vision and image analysis. This has raised considerable interest in the medical imaging community, where such approaches have been shown to provide value in a large number of clinical applications [81]. While DL approaches are useful for a number of tasks in medical imaging, we limit the brief overview below to the application of DL for the classification of conventional MG. For a detailed discussion and a review of recent publications, please refer to Hamidinekoo et al. [82].

On MG images, a tumor may appear as mass, through the presence of microcalcifications or through distortion of tissue. These features can readily be detected through feature extraction by dedicated conventional CAD systems (see above), but these approaches have been challenged through the rise of DL approaches for classification. In a recent study [83], a DL system was trained and evaluated on a large dataset of approx. 45,000 images; the system's performance to detect cancer was compared against a state-of-the-art MG CAD system and against human readers. This study found that the CNN approach outperformed the CAD system at low sensitivity and yielded comparable results at high sensitivity; there was no significant difference in performance between the CNN and human readers.

A critical requirement for training DL algorithms is the availability of sufficient amounts of training data, where the relevant clinical endpoint is available as label to each dataset. This has been a considerable limitation of the application of DL algorithms in medical imaging, which has recently been mitigated through several challenges, where large amounts of labeled data were made available to the public; participants in these challenges trained their algorithms on the public data and submitted them to the competition. For an unbiased evaluation of these trained algorithms, their performance was determined on a held-back validation dataset, which determined the position of the algorithm on the so-called private leaderboard. In the field of MG, an influential challenge was the DREAM challenge which aimed to assess algorithms for risk stratification in screening MG [84]. In this challenge, 86,000 exams were made available, together with a binary label for each image, stating whether BC was diagnosed within 12 months after diagnosis. The winning algorithms achieved AUC values of 0.874 and 0.843, respectively. The second-best algorithm [85] not only predicted the diagnosis of BC on the breast level, but it was also able to localize and detect lesions. As a detector system, this algorithm achieved high sensitivity with very few false positive marks.

Outlook and Perspectives

Imaging methods become more and more complex, promising individualized treatment options arise, and the number of BC patients is increasing – this highlights the crucial role of correct assessment and interpretation of imaging data. At the same time, the workload of experienced breast radiologists increases and the recruitment of new specialists becomes difficult.

Whereas early CAD systems have focused on the mere detection of lesions, recent advances in radiomics and DL promise not only to detect, but also a complete lesion characterization, including tumor type specification and prognosis. Modern CAD systems have reached a reading level that is comparable to human readers, and radiologists should consider options and modalities how these systems can be incorporated into clinical care, especially in a screening setting – for example, CAD systems might replace a second human reader (they reached the same sensitivity as average screening readers in a retrospective study [86]) or preselect normal images to reduce the readers' workload [87].

In addition, much more information about the type of lesions we detect in our patients, the heterogeneity of tumors, and the differences in treatment response and prognosis is inherited in our images and has so far maybe been hidden to the human reader's eyes or is too complex to be completely assessed by the human read-only. In the past, CAD tools helped mainly to detect lesions but did not add much more information for their interpretation. With more advanced diagnostic techniques, like radiomics and DL, a translation from imaging extracted features into tumor-type specifications or prognosis prediction will come more and more into focus.

Computer systems may also be helpful for risk assessment and stratification in high-risk patients, taking into account breast density, tissue structure, background enhancement, or other image features; this is an active area of research. In this sense, the replacement debate loses momentum; instead, radiologists should appreciate and embrace the added value that artificial intelligence systems provide – quite possibly, these systems will relieve radiologists from routine work, will improve the level of patient-centered care, and, maybe most importantly, will enable the radiologist to focus on human interaction and individual treatment.

Disclosure Statement

The authors have nothing to disclose.

Funding Sources

No funding was provided for finalizing this paper.

Author Contributions

All authors of this paper contributed to the structure of the paper, were active in literature review, and careful correction of the draft.