Heterogeneity Measurement of Cardiac Tissues Leveraging Uncertainty Information from Image Segmentation

Abstract

Identifying arrhythmia substrates and quantifying their heterogeneity has great potential to provide critical guidance for radio frequency ablation. However, quantitative analysis of heterogeneity on cardiac optical coherence tomography (OCT) images is lacking. In this paper, we conduct the first study on quantifying cardiac tissue heterogeneity from human OCT images. Our proposed method applies a dropout-based Monte Carlo sampling technique to measure the model uncertainty. The heterogeneity information is extracted by decoupling the intra/inter-tissue heterogeneity and tissue boundary uncertainty from the uncertainty measurement. We empirically demonstrate that our model can highlight the subtle features from OCT images, and the heterogeneity information extracted is positively correlated with the tissue heterogeneity information from corresponding histology images.

1. Introduction

Arrhythmia is a major type of cardiovascular disease that afflicts millions of patients in the United States [17]. A standard intervention to treat arrhythmia is radio-frequency ablation (RFA), an intra-cardiac procedure that directs a catheter and delivers heat to areas where irregular rhythms are observed. Current clinical guidance of RFA is based on low-resolution imaging modalities with limited tissue composition information [26]. For many patients, an additional procedure is required to achieve a chronic successful termination of the arrhythmia. Thus, precise intervention is challenging. Knowledge of patients’ heart structure could help to optimize the intervention strategy by identifying arrhythmia substrates and avoiding critical structures. In addition, previous research has suggested that heterogeneity within the myocardium, such as fibrosis and adipose, are substrates that are potential mechanisms for the generation and maintenance of arrhythmias [1,11,28]. Thus, it is important to identify arrhythmia substrates and quantify their heterogeneity.

Optical coherence tomography (OCT) [14] is a depth-resolved imaging modality that can provide micron-level-resolution images in real time. It can detect micro-structures within cardiac tissues [4,8–10,21] and has great potential for precise guidance of RFA. Some forward-viewing OCT catheters have been designed to image cardiac tissues while the catheter is in contact with the cardiac surface [5,30]. As an interferometry imaging system which highlights the differences of reflective index of adjacent tissues, OCT can depict boundaries among tissue layers, opening a great possibility to segment various tissue compositions using either conventional machine learning or deep learning methods. However, quantitative analysis of heterogeneity within cardiac OCT images is still lacking.

Cardiac tissue heterogeneity refers to intermittent and spatially-varying structural distributions within the myocardium [11], or adipose-infiltrated fibrotic myocardium [28]. Histology heterogeneity assessment is used as a reference for cardiac fibrosis analysis, according to the conventional approach considering histology imaging as the gold standard [20]. OCT imaging provides an alternative non-invasive imaging solution, but quantification of heterogeneity in cardiac OCT images remains particularly challenging. First, unlike histology images, the cellular walls of cardiac muscle cells are not visible in OCT images. Second, there are subtle differences in pixel intensity and texture contrasts between normal myocardium and fibrotic myocardium. Third, OCT images are in grayscale, which have less tissue information than colored histology images. Other imaging modalities have a too low spatial resolution to identify cardiac fibrosis heterogeneity. These modalities rely upon indirect measures to assess cardiac tissue, such as mechanical deformation, strain, and wall thickness [2], which are only proxies to quantify cardiac fibrosis heterogeneity.

In this paper, we propose a novel deep learning framework for cardiac tissue heterogeneity measurement. We evaluate our method by comparing the heterogeneity information from OCT images with its corresponding histology images heterogeneity. In summary, this paper has following contributions:

1. This is the first paper, to the best of our knowledge, that attempts to extract the tissue heterogeneity information from OCT images via deep learning-based uncertainty measurement.

2. We empirically demonstrate that our model can highlight subtle features from OCT images, offering a way to extract the tissue heterogeneity information in a non-invasive way for real-time imaging and processing.

2. Methodology

Our goal is to measure the heterogeneity of human cardiac tissue on OCT images. As illustrated in Fig. 1, our proposed model consists of three major modules: 1) robust learning on region-based labels, 2) uncertainty measurement, and 3) heterogeneity information extraction. In the robust learning module, we use focal loss [19] to robustly learn the representative features from the region-based labels. Then, a novel dropout-based Monte Carlo sampling technique is applied for uncertainty measurement to get the uncertainty map of the prediction. Finally, the tissue heterogeneity information is extracted by decoupling the intra/inter-tissue heterogeneity and tissue boundary uncertainty from the uncertainty maps.

Fig. 1.

The framework of our proposed algorithm in testing. For each image, an uncertainty map is generated by the comparison of predictions from the network with dropout layers on or off. Based on the uncertainty map, a separate purity map is calculated to highlight the tissue heterogeneity information.

2.1. Segmentation Framework

We use the ReLayNet [24] architecture as the base of our network. ReLayNet is an end-to-end image segmentation framework that achieves state-of-the-art performance. Inspired by [16], we add dropout layers in the three inner encoder blocks in the training phase. The introduction of dropout addresses the overfitting issue for small datasets and also lays a foundation for uncertainty quantification in the next section. The network is jointly optimized by the following loss function:

$$L=w\cdot DL+FL$$ (1)

where w is a trade-off parameter. FL is the focal loss [19] and DL is the Dice loss defined as:

$$DL=1-\frac{1}{K}\sum _{k=1}^K\frac{2\sum _{x\in \Omega }\left(p_k(x)q_k(x)\right)}{\sum _{x\in \Omega }{\left(p_k(x)\right)}^2+\sum _{x\in \Omega }{\left(q_k(x)\right)}^2}$$ (2)

where p_k(x) is the predicted probability of class K at the pixel position x ∈ Ω with $\Omega \subset {\mathbb{Z}}^2$, q_k(x) is the one-hot ground-truth label, and K is the number of classes. Based on Dice coefficient, Dice loss evaluates pixel-wise agreement between the prediction and the ground truth.

2.2. Uncertainty Measurement

We use dropout [27] based Monte Carlo sampling technique [7,13,26] for uncertainty measurement. For each segment, a pixel-wise uncertainty map is generated to present the posterior distribution for the tissue prediction map. During the training process, the dropout layers are opened to robustly learn the discriminative features. At the test time, as shown in Fig. 1, they are turned off to provide a baseline prediction pred. Then, served as proximal inference, these layers are turned on again N_it times to get the Monte Carlo sampling maps predⁱ_MC, i ∈ {1, 2, …, N_it}. For each pixel position x ∈ Ω, the uncertainty map UM is generated as:

$$UM(x)=\frac{1}{N_{it}}\sum _{i=1,\dots ,N_{it}}\left({\mathbf{1}}_{\left\{pre{d}_{MC}^i(x)\ne \mathit{\text{pred}}(x)\right\}}\right)$$ (3)

where 1_A is the indicator function of A. The value in the uncertainty map encodes the empirical confidence level of the network on its prediction. More specifically, a high uncertainty value on a pixel indicates a highly unreliable prediction, which corresponds to a low confidence.

2.3. Heterogeneity Measurement

In cardiac OCT images, the uncertainty of prediction comes from the intra/inter-tissue heterogeneity and the boundary effect. The boundary effect is caused by the subtle boundaries between two different tissue types. To remove this unwanted boundary effect, we generate a purity map by multiplying the uncertainty map with a boundary weight mask. This boundary weight mask is generated from the tissue prediction map via assigning smaller weights on the boundary. Similar to the weighting map in [24], lower weights are assigned to pixels at the boundary and higher weights are assigned to pixels in the central region of a tissue. Therefore, this boundary mask highlights the tissue boundary regions, decoupling the tissue heterogeneity and the boundary effect from the uncertainty map. Hence, the purity map is more informative to evaluate the tissue heterogeneity features. The whole process to generate a purity map is presented in Fig. 1.

Our tissue heterogeneity information is measured by homogeneity values [3]. Following previous research [3,6,15,29], the homogeneity values of both purity maps and tissue images (histology images) are calculated from gray-level co-occurrence matrix (GLCM). In particular, we average four directions (0°, 45°, 90° and 135°) to make the descriptor rotation invariant [15]. Then, we extract the homogeneity from the GLCM descriptors by using the following equation [3,6]:

$$\mathit{\text{Homogeneity}}=\sum _{i,j}\frac{G(i,j)}{1+|i-j|}$$ (4)

where G(i, j) represents the (i, j) value of the GLCM. The homogeneity value can be used to measure the closeness of the distribution of pixel intensities in the GLCM [18], representing the amount of local variations in an image [3]. We compare the homogeneity values of purity maps with those of histology images. A high similarity value, which is positive correlation, between values from these two groups indicates that our purity maps can extract the tissue heterogeneity information.

3. Experimental Results

3.1. Dataset and Experimental Setup

We acquired an in-vitro cohort of 185 images taken from 15 human atria and ventricles from the Thorlabs OCT system. The samples were acquired through an National Disease Research Interchange (NDRI) approved protocol from Columbia University [8]. Upon imaging, sections of samples were stained with Masson’s Trichrome. We used white-light images acquired simultaneously with the OCT images to guarantee that histological slides and OCT volumes are from the same regions. This, along with inking, was used to match OCT with histology images. Each OCT image is of size 512 × 600 or 512 × 800 pixels with a field of view of 2.51 mm × 4 mm. Based on histology and the guidance from a pathologist, two investigators labeled the OCT images into the following tissue types: endocardium, myocardium, artifacts, adipose & fibrosis, and other tissue types. These investigators were blind to our algorithm results.

In order to fully utilize our dataset, we perform 6-fold cross-validation in evaluation. The images are randomized by donors. We have two pre-processing steps: first, we crop each OCT image into a depth of 360 pixels (~1.76 mm); next, we partition the images into a set of overlapping patches with a size of 360 × 64 pixels for data augmentation.

3.2. Quantitative Results

Learning from Region-Based Labels:

We empirically set the epoch = 500, learning rate = 0.001, and w = 0.5 (Eq. 1). Then, we use both Dice coefficient (DC) and accuracy (AC) to evaluate the segmentation performance of our proposed model. Table. 1 compares the experimental results averaged over the 6-fold cross-validation sets with the baseline algorithm, RelayNet [24], based on a similar network structure but without dropout and focal loss. However, these evaluation metrics cannot fully reflect the segmentation performance. The manual labels are region-based, which is not accurate enough for pixel-wise evaluation. Among six validation sets, the best accuracy is above 0.8. In addition, compared with the baseline algorithm, the use of focal loss and dropout layers improves both DC and AC. These results demonstrate that our network has the ability to robustly learn representative features from region-based labels.

Table 1.

Comparison of segmentation results

Method	Accuracy (AC)	Dice coefficient (DC)
Proposed method	0.726 ± 0.08	0.605 ± 0.04
RelayNet architecture	0.698 ± 0.10	0.596 ± 0.03

Visual results demonstrate the learning ability of our model on region-based labels. Figure 2 illustrates the predicted tissue maps, on two myocardium examples with some adipose and fibrosis regions. Our algorithm pinpoints isolated fibrosis in Fig. 2(c) and the large adipose region in Fig. 2(g). Overall, the prediction results are consistent with both OCT images and histology findings.

Fig. 2.

Segmentation results from the proposed method on two myocardium regions. (a, e) original OCT images; (b, f) manual labels; (c, g) prediction results; (d, h) corresponding histology images. Arrows highlight locations of excellent agreement between our tissue prediction results and histology images. Scale bar: 500 μm

Uncertainty Measurement:

We set N_it to 10, which is the smallest number leading to a stable uncertainty map. In Fig. 3, we compare an uncertainty map with its histology to show the effectiveness of our uncertainty measurement. In concordance with the results in [16], we observe that the uncertainty map has a high value at the tissue boundaries and subtle features that are hard to visualize. Therefore, the homogeneous regions appear low uncertainty to the model. These results visually demonstrate that our proposed algorithm can provide accurate localization of model uncertainty and this uncertainty is related to the tissue heterogeneity.

Fig. 3.

An uncertainty map from a human myocardium region. (a) the original OCT image; (b) the uncertainty map; (c) the corresponding histology image. The uncertainty map highlights heterogeneous regions caused by fibrosis and adipose. Scale bar: 500 μm.

Heterogeneity Measurement:

The positive correlation between purity map heterogeneity and the tissue heterogeneity can be observed in the scatter diagram. Figure 4 shows the normalized heterogeneity results on a cross-validation set with 95% confidence interval. The homogeneity from the purity map has the strongest correlation relationship with tissue homogeneity. The correlation coefficient obtained on these testing images is 0.7117, which achieves the highest value among other methods (OCT images: 0.3501, Uncertainty maps: 0.5959). From the interpreting table in [22,25], the homogeneity values obtained from the purity maps have a high positive correlation with the homogeneity values obtained from the histology images. Therefore, tissue boundary removal is essential to highlight purity map information within tissue regions. In addition, from the results of 95% confidence interval, using a purity map for heterogeneity measurement has less outliers. Hence, the purity map highlights the tissue heterogeneity information and it has the ability for tissue heterogeneity measurement.

Fig. 4.

Scatter diagram for normalized homogeneity values. (a) homogeneity values of OCT images vs. histology images; (b) homogeneity values of uncertainty maps vs. histology images; (c) homogeneity values of purity maps vs. histology images. Compared with the OCT images and uncertainty maps, homogeneity obtained from purity maps has the highest correlation with tissue homogeneity.

Our algorithm shows good performance on quantifying both inter-tissue heterogeneity and intra-tissue heterogeneity. Figure 5 shows two purity maps from different human hearts, demonstrating two scenarios of intra/inter-tissue heterogeneity. A high value in the purity map indicates a high heterogeneity and a low value indicates a high homogeneity. In Fig. 5 (a)–(c), the purity map has low values (high homogeneity) all through the homogeneous cardiac regions. Meanwhile, it highlights heart fibers that are not well aligned due to the dilation of the myocardium. In Fig. 5 (d)–(f), two clusters with high heterogeneity are enhanced (arrow and double arrow). The purity map successfully highlights these subtle features in the OCT images. This intra/inter-tissue heterogeneity information is entirely determined by our purity maps, which cannot be observed directly from the labels since all manual labels are region-based.

Fig. 5.

Purity maps from human myocardium regions. (a, d) original OCT images; (b, e) purity maps; (c, f) corresponding histology images. The purity maps highlight the heterogeneity within the human myocardium regions. Scale bar: 500 μm.

Cardiac heterogeneity measurement from OCT images is very challenging and complicated. Figure 6 shows two representative outlier cases from Fig. 4 (c). In Fig. 6 (a)–(c), the adipose regions in the purity map are more heterogeneous than the histology images. It is caused by the fixation and dehydration in the histology process. During the histology process, scattering changes and tissue architectural distortion are inevitable due to the shrinkage of epithelial, muscle, and connective tissue layers [12]. Structural information may get lost after the shrinkage, especially in highly heterogeneous regions. Therefore, a lower value of heterogeneity is observed, leading to a low correlation value. In addition, stain variation in histology could also result in a lower correlation value. In Fig. 6, the loose and dense collagen regions in (a)-(c) are purple and blue, while in (d)-(f), they are pink and pale with isolated light blue strands. This is due to the concentration of the stains and the timings of staining [23]. This issue may be mitigated by improving the histology protocol with better control on the amount of stain used in processing.

Fig. 6.

Examples of two outliers from Fig. 4 (c). (a, d) original OCT images; (b, e) purity maps; (c, f) corresponding histology images. Scale bar: 500 μm.

4. Conclusion

In this paper, we propose the first deep learning framework for cardiac tissue heterogeneity measurement on OCT images. Our proposed algorithm consists of three powerful modules: robust learning on region-based labels, uncertainty measurement, and heterogeneity information extraction. We show that these modules are necessary and benefit from each other. Our results indicate that the purity maps can successfully highlight the subtle features from the OCT images and they are highly consistent with the corresponding histology images. Combining with the tissue prediction map, the purity map could be used for the guidance of RFA intervention by identifying regions that have diffusive adipose-fibrosis or heterogeneous myofiber alignment. In the future, we will evaluate our method on a larger dataset and further improve our model in 3D.