Characterizing breast lesions through robust multimodal data fusion using independent diffuse optical and x-ray breast imaging

Abstract.

To enable tissue function-based tumor diagnosis over the large number of existing digital mammography systems worldwide, we propose a cost-effective and robust approach to incorporate tomographic optical tissue characterization with separately acquired digital mammograms. Using a flexible contour-based registration algorithm, we were able to incorporate an independently measured two-dimensional x-ray mammogram as structural priors in a joint optical/x-ray image reconstruction, resulting in improved spatial details in the optical images and robust optical property estimation. We validated this approach with a retrospective clinical study of 67 patients, including 30 malignant and 37 benign cases, and demonstrated that the proposed approach can help to distinguish malignant from solid benign lesions and fibroglandular tissues, with a performance comparable to the approach using spatially coregistered optical/x-ray measurements.

Diffuse optical tomography (DOT) has been used increasingly in breast cancer research for characterizing breast lesions and monitoring treatment responses.^1,2 Standalone optical breast imaging has been shown to be effective when characterizing large lesions with known locations;³ however, due to its intrinsic low spatial resolution, DOT imaging becomes challenging in applications where the target tumors are relatively small with unknown types and uncertain locations. Therefore, an increasing number of DOT breast imaging systems have been built to utilize another imaging modality,⁴ predominantly a structure-oriented one, such as magnetic resonance imaging⁵ or ultrasound,⁶ to provide structural priors under a multimodal imaging scheme. Among these efforts, research on combining spatially coregistered DOT with x-ray mammography—the current clinical standard for early cancer diagnosis—for joint imaging analysis has shown promise in differentiating malignant from benign tumors,⁷ suggesting the potential of DOT in assisting breast cancer management in early stages. We conducted our initial investigation using an integrated DOT and three-dimensional (3-D) mammography system—digital breast tomosynthesis (DBT).^7,8 However, a generic method that permits synergistic use of separately acquired optical and x-ray images, accommodating both two-dimensional (2-D) and DBT mammography systems, is more attractive due to the potential for accelerating the clinical adoption of breast DOT and simplifying its regulatory pathways.

While the ideas of taking independent measurements and combining results in postprocessing have been explored,^4,9 the proposed approach is innovative in several aspects. First, it seeks to simultaneously use the information acquired from the two modalities in a single diagnosis, enabled by our structure-guided DOT imaging algorithm,¹⁰ whereas previous works of combining independent imaging information were mostly for validating standalone optical imagers.^9,11 Second, our optical breast scanner is designed to mimic standard mammographic compressions to minimize the registration error. In other similar works, the optical scans were acquired with independent settings without the intent to match the structural scans.⁹ Furthermore, the focus of this research is to enable robust image fusion with x-ray mammography, thus specifically addressing the clinical need for improving early diagnosis of breast cancer. There is no similar previous research that explores this combination and addresses the same clinical challenge.

To demonstrate the feasibility of combining DOT with independently acquired mammograms in this study, we retrospectively investigate a subgroup from a previously acquired clinical population ($N=450$) using our combined DOT/DBT system.^7,8 The study protocol (#1999P010998) was approved by the Massachusetts General Hospital Institutional Review Board, and written informed consent was obtained from all subjects. For each patient, we acquired spatially coregistered bilateral breast DOT and DBT scans. In addition, most subjects obtained 2-D diagnostic mammograms on a different x-ray scanner during a separate visit. Because the DOT measurements and the 2-D diagnostic mammograms were acquired separately, the combination of such measurements provides us a realistic dataset with which we can test our hypothesis, i.e., an independently measured mammogram can guide DOT image reconstruction for accurate characterization of breast tumors. Meanwhile, the additional 3-D DBT breast scans, coregistered with the DOT data, can be readily used as a reference to evaluate the accuracy of both registration and prior-guided optical reconstructions.

Among the 450 recruited patients in our previous study, 67 patients were selected for this study using the following criteria: (1) bilateral mediolateral-oblique view 2-D diagnostic mammograms are available within 3 months before or after the DOT scan; (2) valid bilateral optical measurements are available; (3) one of the breasts must contain a malignant or benign lesion of size $\ge 8\mathrm{mm}$, identified by a senior radiologist, and the contralateral breast must be normal. Among the 67 lesion cases, 30 were malignant (confirmed by biopsies), 24 were solid benign, and 13 were cystic lesions.

The first step to analyzing separately collected optical and mammographic measurements is to register the two image spaces. For this purpose, we have developed a flexible contour-based registration workflow. For this particular dataset, we use the tissue–air boundaries extracted from the DBT to represent the breast shape because they are coregistered with the DOT data. To register the separately acquired 2-D mammogram with this breast surface, we extracted the contour lines from both the mammogram as well as the middle-axial cross-section of the breast surface and then matched the two contours (Fig. 1) using an automated two-step optimization process. First, we perform a coarse registration based on matching three “landmarks”—the nipple and two equal-distance endpoints traversing toward either direction of the breast contours starting from the nipple position (see markers in Fig. 1). Second, we perform an iterative closest point¹² search seeking to minimize the Euclidean distance between the two contours under an affine transformation.

Fig. 1.

(a) Middle-axial slice of the digital breast tomosynthesis (DBT) scan; (b) 2-D mammogram of the same breast; (c) checker-box ($1\times 1{\mathrm{cm}}^2$) overlay of (a) and (b) after registration; (d and e) zoom-in views; and (f) box-plots of the registration lesion offsets (left) and lesion sizes (right). Mammogram breast contours before (blue solid) and after (blue dashed) registration are shown and compared with the reference (red).

In Fig. 1, we show a representative registration result from a breast containing an 11-mm solid benign tumor (pointed by arrows). Despite the obvious differences in contrast and intensity between the 2-D and reference DBT images, the internal tissue structures generally align well after registration [Figs. 1(c)–1(e)] with clear continuation of the vessels and fibrous tissues readily observable. For all 67 cases, a mild ($12.7\pm 0.9\mathrm{mm}$) lesion offset was observed after registration [Fig. 1(f)], which is comparable to the lesion size distribution ($14.9\pm 0.8\mathrm{mm}$).

Once we map the 2-D mammogram to the breast model conforming to the DOT measurements, we can then apply our previously developed structural-prior-guided DOT image reconstruction¹⁰ to utilize the anatomical information in a joint optical/x-ray imaging analysis. To do this, we applied a “dual-Gaussian” segmentation algorithm¹³ to the 2-D mammogram to derive estimated volume fractions of two normal breast tissue compositions, namely adipose and fibroglandular tissues.¹⁰ These 2-D tissue compositional maps were then vertically stacked repeatedly to form a 3-D volume and mapped onto the breast mesh using the aforementioned registration transformation. For comparison, we also ran DBT-guided reconstructions. We applied histogram equalization to the DBT image first before converting it into structural priors using a “threshold” approach.¹³

Using the mammogram-derived adipose and fibroglandular compositional priors, a regularization matrix can be created and subsequently used in a Tikhonov-regularized Gauss–Newton reconstruction algorithm.⁸ In Fig. 2(f), we show the recovered total hemoglobin concentration (HbT) image for a breast containing a 25-mm malignant tumor (pointed by arrows) using the proposed approach. By comparison, in Fig. 2(e), we show the HbT image of the same breast guided by the coregistered DBT scan. In both cases, the malignant tumor can be readily identified with an elevated HbT level as a result of increased angiogenesis around the tumor regions.¹ In comparison, the recovered HbT images on the contralateral normal breast using the same procedure [Figs. 2(g) and 2(h)] show a lower HbT in the mirrored region. Moreover, as shown in the lower panels of Figs. 2(e)–2(h), using 2-D priors [Figs. 2(f) and 2(g)] yielded similar imaging features in the sagittal views comparing to the 3-D prior-guided reconstructions [Figs. 2(e) and 2(h)]. This result indicates that parallel-plate DOT systems^8,9 are relatively insensitive to the structural priors in the orthogonal/sagittal directions.

Fig. 2.

(a,d) Axial slices of the DBT and (b,c) 2-D mammograms of a breast containing a 25-mm malignant tumor (a,b) and its contralateral normal breast (c,d), respectively; (e–h) the recovered total hemoglobin concentration (HbT, $\mu \mathrm{M}$) images in axial (top) and sagittal views (bottom) using adipose and fibroglandular tissue compositional priors derived from the corresponding x-ray images in the top row. The true (black) and mirrored (green) tumor regions-of-interest are marked using dashed circles. The sagittal cross-sections are marked by horizontal dashed lines in the axial views above.

To statistically assess the capabilities of x-ray prior-guided breast DOT using either postregistered or coregistered structural priors in delineating breast tumors from a heterogeneous tissue background, we first defined a tumor profile in the form of a normalized Gaussian sphere¹³ with a full-width half-maximum of 10 mm centered at the lesion location. Using this tumor profile, together with previously derived adipose and fibroglandular compositional maps, the “average” optical properties for each tissue type can be extracted using weighted averages. We can then perform pairwise two-tailed $t$-tests between the lesion, adipose, and fibroglandular optical properties within each tumor-bearing breast. Moreover, we further tested our hypothesis of optically differentiating malignant from solid benign and normal tissues by performing two-sample two-tailed $t$-tests between the malignant tumor-bearing breasts ($N=30$) and solid-benign tumor-bearing breasts ($N=24$). To minimize the influence of intersubject variations, we normalized tumor and fibroglandular optical properties by those of the adipose tissue before performing the $t$-tests.⁷

In addition, we applied the developed contour-based registration algorithm to all bilateral mammograms and created mappings that mirror the tumor profile from the tumor side to the normal side, as exemplified in Fig. 2(g). Note that there is no real lesion in the contralateral breast. The mirrored tumor profile enables us to extract the optical properties from the bilateral breasts and form a pairwise analysis between lesion and normal tissues, which gives us more confidence in the accuracy of the estimated optical properties.

The resulting $p$-values from the aforementioned $t$-tests for 2-D and DBT-guided image reconstructions are summarized in Tables 1 to 3. All statistically significant ($\alpha =0.05$) tests are highlighted in bold; nearly significant tests ($p<0.1$) are marked by “*”.

Table 1.

Comparisons within the same breast (paired $t$-tests). Statistical significance (in $p$-values) between total hemoglobin concentration (HbT) values of different tissue types in two-dimensional (2-D) and digital breast tomosynthesis (DBT)-guided optical reconstructions.

HbT	Adipose versus fibroglandular	Adipose versus tumor	Fibroglandular versus tumor
Malignant (30)	0.003 (0.001)	0.004 (0.002)	0.011 (0.020)
Solid (24)	0.546 (0.416)	0.553 (0.877)	0.463 (0.780)
Cyst (13)	0.108 (0.190)	0.089*(0.209)	0.193 (0.412)

Note: in each $p$-value pair, $p$-value for 2-D guided result is followed by that of DBT-guided result (in parentheses).

Table 3.

Bilateral comparisons (paired $t$-tests).

HbT	fibrograndular versus fibrograndular	Normal versus lesion
Malignant (30)	0.144 (0.550)	0.062* (0.080*)
Solid (24)	0.136 (0.705)	0.555 (0.718)
Cyst (13)	0.854 (0.857)	0.953 (0.999)

Note: in each $p$-value pair, $p$-value for 2-D guided result is followed by that of DBT-guided result (in parentheses).

From Tables 1 to 3, the proposed technique appears to demonstrate a comparable performance to the coregistered DBT-guided DOT. In Table 1, combining separately acquired optical/x-ray data resulted in statistically significant differentiation between all combinations of adipose, fibroglandular, and malignant tissues in the recovered HbT of malignant tumor-bearing breasts. More importantly, as shown in Table 2, the proposed approach can differentiate malignant from solid benign lesions ($p=0.006$) as well as from the normal tissue ($p=0.010$) using normalized HbT. These results are consistent with those derived from the coregistered DBT-guided approach (p-values shown in parentheses), as well as our earlier findings.⁷ The mean HbT of the adipose and fibroglandular tissues of all investigated breasts ($N=134$) is $22.80\pm 0.53\mu \mathrm{M}$ and $22.97\pm 0.53\mu \mathrm{M}$ in 2-D guided images, and $22.65\pm 0.53\mu \mathrm{M}$ and $23.01\pm 0.52\mu \mathrm{M}$ in coregistered DBT-guided images, respectively. A strong correlation ($R^2=0.98$) was also observed between all lesion HbT values recovered using 2-D and DBT-guided approaches. In addition, the comparisons of fibroglandular tissues between the lesioned and contralateral breasts show no significant differences (Table 3). By contrast, tests between the true and mirrored (on the contralateral normal side) malignant tumor regions show near significance in both 2-D ($p=0.062$) and DBT-guided ($p=0.080$) reconstructions. The above results suggest that despite the negative impact resulting from the registration error and the approximation to 3-D anatomy with 2-D structures, the proposed approach shows a comparable performance in separating malignant tumors from solid benign lesions and fibroglandular tissues comparing to the coregistered optical/x-ray approach.

Table 2.

Cross-group comparisons (two-sample $t$-tests).

Malignant versus solid	Malignant versus normal	Solid versus normal
0.006 (0.025)	0.010 (0.007)	0.210 (0.646)

Note: in each $p$-value pair, $p$-value for 2-D guided result is followed by that of DBT-guided result (in parentheses).

In summary, motivated by our past research on integrated and coregistered DOT/DBT imaging of breast lesions, we hypothesized that the optical breast imaging can also utilize a separately acquired x-ray digital mammogram to achieve a comparable performance in characterizing breast lesions. To validate this hypothesis, we retrospectively studied a clinical dataset where separately obtained DOT data and mammograms were available, along with coregistered 3-D DBT scans as reference. We developed a flexible contour-based registration workflow to transform a 2-D mammogram to the DOT imaging domain and incorporated it in our prior-guided optical image reconstruction. While we anticipated reduced accuracy in tumor characterization due to (1) the registration error resulted from separate breast compressions and (2) the approximation of 3-D tissue anatomy by 2-D structures, such degradation did not seem to generate significant deviations from the results using coregistered 3-D tissue priors. From 67 bilateral breast measurements, the malignant tumors’ HbT recovered using separate x-ray priors is significantly higher than those of solid benign and normal breast tissues, similar to our previous findings.⁷ From these tests, we believe that the proposed flexible multimodal data fusion technique can effectively combine the tissue functions from DOT with tissue anatomy from x-ray mammography in breast tumor diagnosis without the complexity of developing a fully integrated imaging system. This approach allows us to develop a portable, low-cost freestanding DOT breast scanner that can be used side-by-side with any one of the over 9000 pre-existing digital mammography systems in the US and make functional diagnosis readily available.

In the next phase, we will continue improving the proposed technique and further enhance the optical image quality and the accuracy of the optical property quantification. We plan to further improve registration of breast anatomy (where so far we used only breast contours) by exploring the use of x-ray fiducial markers. This would increase the number of landmarks and further reduce the registration error. We will also investigate the possibilities of applying less compression in the DOT scan to make the procedure more comfortable for the patients.