A topological encoding convolutional neural network for segmentation of 3D multiphoton images of brain vasculature using persistent homology

Abstract

The clinical evidence suggests that cognitive disorders are associated with vasculature dysfunction and decreased blood flow in the brain. Hence, a functional understanding of the linkage between brain functionality and the vascular network is essential. However, methods to systematically and quantitatively describe and compare structures as complex as brain blood vessels are lacking. 3D imaging modalities such as multiphoton microscopy enables researchers to capture the network of brain vasculature with high spatial resolutions. Nonetheless, image processing and inference are some of the bottlenecks for biomedical research involving imaging, and any advancement in this area impacts many research groups. Here, we propose a topological encoding convolutional neural network based on persistent homology to segment 3D multiphoton images of brain vasculature. We demonstrate that our model out-performs state-of-the-art models in terms of the Dice coefficient and it is comparable in terms of other metrics such as sensitivity. Additionally, the topological characteristics of our model’s segmentation results mimic manual ground truth. Our code and model are open source at https://github.com/mhaft/DeepVess.

1. Introduction

The health of the brain and heart, as the most two critical organs, interconnect through the brain vasculature. Since the brain energy depends exclusively on the supply of oxygen and glucose through the bloodstream and the brain energy reserve is very limited, the brain requires steady and reliable blood perfusion [24]. Therefore, any blood flow interruption causes temporary or permanent functional impairments [38]. Researchers and physicians utilize various imaging modalities, such as multiphoton microscopy (MPM) [4] and optical coherence tomography [39], to study and examine the 3D geometrical, topological, and fluid dynamics characteristics of the brain vasculature, including capillaries in live animal models.

Accurate analysis of brain vasculature over large 3D volumes and multiple subjects within several controlled groups forms a considerable portion of the bottleneck processes in terms of time and personal allocation in biomedical research and medicine. For instance, researchers studied the effects of brain capillary stalling on Alzheimer’s disease in animal models and reported that the image analysis required order of magnitude higher time allocations in comparison to the actual image acquisitions [6, 5, 11, 13, 20]. Notably, the vessel segmentation and vectorization are the prevalent required image analysis tasks [26]. Researchers often consider the segmentation as a classification task and have proposed supervised, unsupervised, and semi-supervised solutions using classical computer vision methods [26] and recently deep learning (DL) methods [27].

In this work, we present a novel method for the segmentation of 3D in vivo MPM images of brain vasculature based on a topological encoding convolutional neural network (CNN) using persistent homology (PH) as an unsupervised loss term in addition to two other supervised loss terms. Our work is motivated by the fact that the-state-of-the-art MPM segmentation methods such as DeepVess [21] suffer from topological errors in the regions with low signal-to-noise (SNR) and by the recent successful integration of the algebraic topology metrics such as persistent homology into differentiable DL layers [7]. Even though persistent homology has been introduced and utilized in various fields such as computational topology, its application in the computer vision is limited to preprocessed feature selections [23]. Our algorithm improved DeepVess’ accuracy significantly, specifically in the low SNR regions, and by producing more plausible results, qualitatively.

2. Related Work

2.1. Vessel segmentation

The segmentation of vasculature networks or particular vessel segments, e.g. the aorta, is an essential task in most biomedical research applications and medical settings due to the fact that the original task requires enormous time and personal allocations. The vessel segmentation problem is tackled through the established computer vision methods, which commonly require feature engineering and accept/reject rules, as well as the DL methods, which are formulated as supervised, unsupervised, or semi-supervised learning problems [14, 29, 42]. For instance, Yi et al. [45] proposed a 3D method using the region growing algorithm utilizing its computational and implementation efficiency. Similarly, Mile et al. [29] represented the segmentation in terms of a deformable parametric model that accurately match the vessel fractal shapes.

On the other hand, due to the current success of DL in various research fields, DL models are popular and yield higher accuracy levels. In recent years, researchers, practitioners, and challenge participants almost exclusively utilize different variants of DL models to tackle different biomedical computer vision problems. For example, a model based on the transfer learning of the Google Inception V3 network [40] surpassed the accuracy of seven ophthalmologists combined in the diabetic retinopathy detection task within the color fundus images [18]. The CNN models, such as the model by Wu et al. [44] can segment vasculature networks successfully, and in different circumstances, recurrent neural network (RNN) models are proposed in place of post-processing conditional random fields (CRF) models [16, 15]. The most common biomedical image segmentation DL architecture theme is called U-Net [35], which is a fully convolutional architecture [28] with encoding and decoding layers in addition to skip connections at different spatial scales.

2.2. MPM segmentation

MPM is the standard imaging modality for studying different live animal models enabling longitudinal studies (e.g. weeks and months) while providing high temporal resolutions (e.g. 2–30 Hz) and sub-micron spatial resolutions with minimally invasive procedure requirements. In comparison to the linear process-based counterpart imaging modality, the intensity of the non-linear processes in MPM imaging (e.g. two-photon excited fluorescence) is proportional to the square or cube of the required average laser power. This non-linearity increases imaging depth and decrease photobleaching [25].

Tsai et al. [42] introduced the first comprehensive MPM image processing software suit, a.k.a. VIDA, based on traditional computer vision methods with the ability to handle large datasets. On the other hand, Teikari et al. [41] utilized 2.5D CNN architecture with CRF post-processing to segment MPM vessel images, while Bates et al. [3] used RNN architecture to extract the vasculature graph representations. Recently, DeepVess et al. [21] was proposed, which is an optimized CNN architecture with a customized loss function and accuracy comparable to that of expert human annotators for both vessel segmentation and centerline extractions and Gur et al. [19] developed an unsupervised neural network with active contours mimicking loss term.

2.3. Topological data analysis

Topological data analysis (TDA) is a mathematical field that studies complex datasets from the algebraic topology point of view. PH, as a TDA theory, is able to characterize high-dimensional manifolds within datasets such as graph-based characterization, unsupervised feature engineering, and image manifold analysis [12, 33, 36].

In comparison to other models with no regularization or preset boundary topologies (e.g. multivariate regression model and support vector machines), Chen et al. [8] used PH to regularize the decision boundaries and to simplify the boundaries. Similarly, Rieck et al. [34] used the PH as a complexity metric for DL architectures by modeling the network as a weighted stratified graph to facilitate the underling optimization problem required for the learning process such as the mimicking the best DL regularization tricks (e.g. dropout and batch normalization) and optimization practice such as validation-based early stopping.

The feature engineering based on the input image is the primary utilization of PH for the segmentation task. Qaiser et al. [32] trained a CNN based on the extracted topological features using enhanced persistent homology profiles to segment tumors in Hematoxylin and Eosin stained histology images. Differently, Assaf et al. [2] segmented the cells by selecting the target object within the candidates based on the largest life span using PH of the cubical complex. Similarly, Gao et al. [17] used the PH to segment the high-resolution CT images of the left ventricle including the papillary muscles and the trabeculae by detecting the salient object with the desirable topological characteristic, i.e. handles, to be restored in the final results. Furthermore, Clough et al. [10] tackled the same problem with the exception of using the PH for providing the gradient signals to the learning optimizer.

3. Method

3.1. Persistent homology

The homology in the context of abstract algebra uses invariant theory to maps the groups on an algebraic variety such as Euclidean spaces to the topological space. If we describe the manifolds in a D-dimensional Euclidean space $\mathcal{X}\in {ℝ}^D$ as a simplicial complex $\mathcal{K}$, which is a generalization of graphs in the higher dimensions, simplicial homology represents the connectivity of $\mathcal{K}$ in terms of homology groups using matrix reduction algorithms. A k-dimensional simplex is the convex combination of k +1 vertices. The vertices correspond to points in a sufficiently nice ambient ${ℝ}^D$ space, which results in the existence of the simplex geometrical realization within $\mathcal{X}$. A collection of simplices forms a simplicial complex $\mathcal{X}$ if for every simplex $\tau \in \mathcal{X}$, every face σ ⊆ τ is also in $\mathcal{X}$ and also for any two simplices τ_i and τ_j, σ = τ_i ∩ τ_j is a face of both τ_i and τ_j.

Homology groups $\left\{H_k(\mathcal{K}):k\in \left\{0,\dots ,D-1\right\}\right\}$ describe the k^th order topological features, i.e. holes of the k^th dimension. For instance, $H_0(\mathcal{K})$, $H_1(\mathcal{K})$, and $H_2(\mathcal{K})$ correspond to connected components, 2D hole or 3D tunnel, and 3D voids, respectively. The Betti numbers (β_i) summarize the homology group information by measuring the cardinality of the $H_i(\mathcal{K})$. For example, a 2D ring has Betti numbers of (1, 1), a filled 2D square has Betti numbers of (1, 0), a torus has Betti numbers of (1, 1, 1), and a sphere has Betti numbers of (1, 0, 1).

Betti numbers are very abstract representation of the homology groups and they are not smooth in noisy datasets. Persistent homology resolves this shortcoming of Betti numbers by measuring the topological features of $\mathcal{X}$ at different resolutions to promote the detection of the true features embedded in $\mathcal{X}$ and exclusion of features constructed based on quantification and noise errors. The simplicial complexes can be ordered into a nested sequence by considering a non-decreasing function $f:\mathcal{K}\to ℝ$ that induces a homomorphism on the simplicial homology groups for each dimension. For every $a\in ℝ$, there is a subcomplex of $\mathcal{K}$ as a sublevel set $\mathcal{K}(a)=f^{-1}(-\mathit{\text{inf}},a]$ and a set of scale weights a₀ ≤ a₁ ≤ … ≤ a_n (a.k.a. filtration parameter) put the $\mathcal{K}$ in filtration based on the ordering of f to represent the growth of $\mathcal{K}$ by changing the filtration parameter (a_i),

$$\varnothing ={\mathcal{K}}_0\subseteq {\mathcal{K}}_1\subseteq \dots \subseteq {\mathcal{K}}_n=\mathcal{K}.$$ (1)

Over the course of $\mathcal{K}$ growth by filtration, topological features are born and die at different scales. Persistence homology maps the birth and death of features into ${ℝ}^2$ space as (b_i, d_i) points, where b_i, d_i ∈ [0, 1] or any other range. The collection of all points based on k-dimensional topological features generate the k^th persistence diagram,

$$\mathcal{P}{\mathcal{D}}_k:(\mathcal{X},f)\to \left\{\left(b_i,d_i\right):i\in {\mathcal{I}}_k\right\}.$$ (2)

If homology is computed over a field, the homology of a filtration admits an interval decomposition, which is the sum of the elements of rank one within the interval [b_i, d_i). The persistence of each point in each $\mathcal{P}{\mathcal{D}}_k$ is the lifetime of the feature and defined as pers(b_i, d_i) = |d_i − b_i|. Furthermore, the points in each $\mathcal{P}{\mathcal{D}}_k$ are indexed in a descending order based on their persistence value. Therefore, the first point in ${\mathcal{I}}_k$ is the most persistent feature and the last one is probably an artifact.

To implement the PH as a differentiable DL layer, the complete inverse map from $\mathcal{P}{\mathcal{D}}_k$ to the input dataset $\mathcal{D}:\text{(}X\text{,}Y\text{)}$ and a loss function are required. The complete inverse map has two steps: mapping $\mathcal{P}{\mathcal{D}}_k$ to $\mathcal{X}$ and then mapping $\mathcal{X}$ to $\mathcal{D}$. The inverse map between $\mathcal{P}{\mathcal{D}}_k$ and $\mathcal{X}$ can be defined by searching for the pair of two simplices τ and τ′ such that b_i = f (τ) and d_i = f (τ′),

$$\pi \left(\mathcal{P}{\mathcal{D}}_k,i\in {\mathcal{I}}_k;f\right):\left(b_i,d_i\right)\to \left(\tau ,{\tau }^{\prime }\right).$$ (3)

The second step of the complete inverse map is to connect the $\mathcal{X}$ to the input dataset $\mathcal{D}$. Since in most computer vision problems, the $\mathcal{D}=(X,Y)$ consists of 3D digitized images as voxel grids, we define the filtration function on the voxel intensity value and simplex based on the voxel neighborhood. Therefore, the filtration is equivalent of level sets at different intensity thresholds and the mapping associates each simplex (τ) to the indices of voxels σ ∈ τ that their intensity values are equal to the filtering cutoff value (a_i),

$$\omega (\tau )=\mathit{\text{argmax}}{}_{\sigma \in \tau }f(\sigma ).$$ (4)

The PH cost functions should represent the distance between the target persistence diagram $\mathcal{P}\mathcal{D}$ and the predicted persistence diagram $\widehat{\mathcal{P}\mathcal{D}}$. We use Wasserstein-1 distance ${\mathcal{W}}_1(p,q)$, a.k.a. Earth-Mover distance, which is related to the minimum cost of stockpile transportation from stock configuration p to stock configuration q, where the transportation cost is equal to the mass times the transportation distance. Note that under a few mild assumptions, ${\mathcal{W}}_1$ is continuous and differentiable almost everywhere [31]. The objective is to minimize the ${\mathcal{W}}_1$ between $\widehat{\mathcal{P}\mathcal{D}}$ and $\mathcal{P}\mathcal{D}$. At first, the cardinalities of $\mathcal{P}\mathcal{D}$ and $\widehat{\mathcal{P}\mathcal{D}}$ are matched by projecting the additional points in $\widehat{\mathcal{P}\mathcal{D}}$ to the diagonal (5) and then the loss function measures the Wasserstein-1 distance between counterpart topological features to equate the number of topological features with high persistent between $\mathcal{P}\mathcal{D}$ and $\widehat{\mathcal{P}\mathcal{D}}$ and push the rest of that features to have zero persistent values,

$$\mathit{\text{proj}}\left(b_i,d_i\right)=\frac{b_i+d_i}{2},$$ (5)

$${\mathcal{E}}_{\mathcal{T}}\left({\beta }_k;{\widehat{\mathcal{P}\mathcal{D}}}_k\right)={\left(\sum _{i=1}^{{\beta }_k}\Vert \left(b_i,d_i\right)-{(0,1)\Vert }^1\right)}^1+{\left(\sum _{i={\beta }_k+1}^{|{\mathcal{I}}_k|}\Vert \left(b_i,d_i\right)-\mathit{\text{proj}}{\left(b_i,d_i\right)\Vert }^1\right)}^1=\sum _{i=1}^{n_k}|1-d_i+b_i|+\sum _{i=n_k+1}^{|{\mathcal{I}}_k|}|d_i-b_i|.$$ (6)

The gradient of the topological loss function (6) with respect to the input image is measurable through the inverse maps (3) and enables the optimization of the whole network in an end-to-end fashion.

An example of this framework is depicted in Figure 1. Panel A and B in Figure 1 are the target and query images, respectively, and the image intensities are normalized into the [0, 1] range. The color blue, red, and dark yellow represent low, medium, and high intensity values such as 0.01, 0.20, and 0.70. In Figure 1.C, green and magenta markers are associated with the target and query images, and square and circle markers are related to H₀ and H₁ homology groups, respectively. The unfilled markers located on the horizontal axis are the elements that are projected to the diagonal line to match the cardinalities of the target persistence diagram $\mathcal{P}\mathcal{D}$ and the query persistence diagram $\widehat{\mathcal{P}\mathcal{D}}$. The arrows are the distances between the target and query features, which measure the distance between the two persistence diagrams. For example, when the filtering value is equal to the red color (e.g. 0.20), the letter “j” will appear, which results in the birth of a hole that is confined between the letter “C” and the letter “j” in addition to birth of a new connected components that corresponds to the dot of letter “j”. The changes due to appearance of letter “j” are reflected in Figure 1.C as the magenta circle and square with mid-range persistent values. The magenta circle and square with low-range persistent values in the lower right corner are related to the letter “Q”.

Figure 1.

An example of a persistence diagram. Panels A and B are the target and query images, respectively. The image intensities are normalized into the [0, 1] range, and the color blue, red, and dark yellow represent low, medium, and high intensity values. In panel C, green and magenta markers are associated with the target and query images, while square and circle markers are related to H₀ and H₁ homology groups, respectively. The unfilled green markers located on the horizontal axis are the projected elements to match the cardinalities of the two persistence diagrams. The arrows in panel C are the distance between the two persistence diagrams.

3.2. Network architecture

We adopted the network architecture of DeepVess [21], which is the state-of-the-art architecture optimized for vessel segmentation in MPM images, as our baseline architecture (Fig. 2). Briefly, DeepVess’ architecture has an input image, which is 7-voxel deep along the third direction and initially apply three 3D convolution layers with the 3 × 3 × 3 kernel size, 32 features, and without padded edges followed by a max-pooling with the 2 × 2 kernel size and 2 × 2 strides. Then two 2D convolution layers with the 3 × 3 kernel size and 64 features followed by a max pooling with the 2 × 2 kernel size and 2 × 2 strides are applied. Finally, the output of the last max-pooling layer is flattened to a fully-connected layer followed by a 1024-node fully-connected layer in addition to a dropout layer at the training time (with 50% probability) and the last fully-connected layer, which is reshaped to the output patch size. LeakyReLU was used as the activation function for all the layers. The width of the input image and consequently the receptive field as the output size were the optimized design hyperparameters that were selected based on the new loss function and prior knowledge of the topological characteristics of brain vasculature networks.

Figure 2.

Our convolutional neural network architecture. The input is a 98 × 98 × 7 3D image patch with a single input channel. The first three 3D convolution layers with the 3 × 3 × 3 kernel size and 32 features were followed by a max-pooling with the 2 × 2 kernel size and 2 × 2 strides. Then two 2D convolution layers with the 3 × 3 kernel size and 64 features followed by a max-pooling with the 2 × 2 kernel size and 2 × 2 strides are applied. Finally, a dense layer classifier with a 1024-node fully-connected hidden layer as well as a dropout layer at the training time was applied. The loss function terms are based on the generalize Dice, cross-entropy, and topological persistence.

3.3. Learning and implementation

We preprocessed the 3D in vivo MPM images of mouse brain vasculature networks following [21]. Briefly, the physiological motion artifacts were removed using non-rigid non-parametric diffeomorphic demons image registration [43], followed by image intensity normalization and 1μm³ spatial resolution scaling.

DeepVess resolved the highly unbalanced class population problem by using a customized cross-entropy loss function (7),

$${\mathcal{L}}_{\mathit{\text{DeepVess}}}(y,\widehat{y})=\sum _{i\in \left\{TP,FP,FN\right\}}\mathcal{C}\mathcal{E}\left(y_i,{\widehat{y}}_i\right),$$ (7)

which is measured over all voxels except true negative voxels. Instead, we used generalized Dice loss function (8) due to the successful results in [30],

$${\mathcal{L}}_{\mathit{\text{Dice}}}(y,\widehat{y})=1-\frac{2{\sum }_i\left(y_i\times {\widehat{y}}_i\right)+ϵ}{{\sum }_i\left(y_i^2+{\widehat{y}}_i^2\right)+ϵ}.$$ (8)

Additionally, we used the normal cross-entropy loss function $\left({\mathcal{L}}_{CE}\right)$ for the initial optimization warm-up to decrease the required training time,

$$\mathcal{C}\mathcal{E}\left(y_i,{\widehat{y}}_i\right)=-y_i\times log\left({\widehat{y}}_i\right)-\left(1-y_i\right)\times log\left(1-{\widehat{y}}_i\right).$$ (9)

In this training regime, initially, ${\mathcal{L}}_{CE}$ is the significant loss term, but after the initial warm-up phase, ${\mathcal{L}}_{\mathit{\text{DeepVess}}}$ will automatically dominate the optimization as a result of the highly unbalanced class populations.

The objective of the topological cost function (6) is to enforce the prior knowledge of the desired topological characterises in terms of the target Betti numbers (β_k). Due to the fractal characteristics of venous and arterial networks and their interaction through the grid networks of capillaries [37]. The topological characterises of brain vasculature network is scale-dependent. Therefore, the optimal size of the field of view based on the topological constrains is the largest field of view, which has similar Betti numbers in various regions. In our case, which is the mouse brain vasculature network, we observed there is no large capillary loop within a 21 × 21 μm² region, which translates to target β₀ of 1 and β₁ of 0, equivalent to one connected component without any hole. Therefore, the topological loss function is deterministic without the need to be inferred from the data, and it can be formulated based on (6) into

$${\mathcal{L}}_{\mathcal{T}}(\widehat{\mathcal{P}\mathcal{D}})=\sum _{k=0}^{D-1}{\mathcal{E}}_{\mathcal{T}}\left({\tilde{\beta }}_k;{\widehat{\mathcal{P}\mathcal{D}}}_k\right)={\mathcal{E}}_{\mathcal{T}}\left(1;{\widehat{\mathcal{P}\mathcal{D}}}_0\right)+{\mathcal{E}}_{\mathcal{T}}\left(0;{\widehat{\mathcal{P}\mathcal{D}}}_1\right).$$ (10)

The final proposed loss function contains three terms: Dice, cross-entropy, and topology loss functions. We measured the topological loss function on 10% of the samples in the mini-batch to reduce the computational cost, while maintaining a sufficient gradient signal,

$$\mathcal{L}(y,\widehat{y})={\mathcal{L}}_{\mathit{\text{Dice}}}(y,\widehat{y})+\lambda {\mathcal{L}}_{CE}(y,\widehat{y})+{\lambda }^{\prime }{\mathcal{L}}_{\mathcal{T}}\left(\mathcal{P}\mathcal{D}\left({\widehat{y}}_1,\mathrm{..},{\widehat{y}}_{\lfloor n/10\rfloor }\right)\right).$$ (11)

We implemented our model in TensorFlow [1] and measure PH using the “Topology Layer” package [7]. We trained our model using Adam stochastic optimization with a learning rate of 10⁻⁴ for 2000 epochs. We set the λ and λ′ to the optimal values of 1 and 0.01 based on the evaluation results of the validation dataset. The division patterns of the image patches within the training dataset were shifted randomly to cover different possible slicing schemes. Also, the same procedure was used to enhance the classification at the inference phase. Our code and model are open source at https://github.com/mhaft/DeepVess.

4. Experiments

4.1. Data

We demonstrate the performance of our algorithm on DeepVess dataset [21], which is the largest, publicly available, manually-labeled, in vivo MPM image dataset of mouse brain vasculature with 24 mice and more than 13 × 10⁶ μm³ labelled images. We utilized their division of the dataset into training, validation, and testing sub-datasets (50%, 25%, and 25%). Similarly, while we fine-tuned and optimized the models based on the performance on the validation dataset, the final model accuracy results were measured on the independent test sub-dataset.

4.2. Baseline and evaluation

We compare our proposed model to the state-of-the-art segmentation methods VesselNN [41], DeepVess [21], and UMIS [19] in addition to VIDA [42] and 3D U-Net [9], which are the methods commonly used in the analysis of MPM and other biomedical image modalities.

We evaluate our model qualitatively with figures in addition to quantitatively based on accuracy, sensitivity, specificity, Dice coefficient (a.k.a. F1), and Jaccard index. This metrics are based on the populations of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) cases,

$$\mathit{\text{Sensitivity}}=\frac{TP}{TP+FN},$$ (12)

$$\mathit{\text{Specificity}}=\frac{TN}{TN+FP},$$ (13)

$$\mathit{\text{Accuracy}}=\frac{TP+TN}{TP+TN+FP+FN},$$ (14)

$$\mathit{\text{Jaccard}}=\frac{TP}{TP+FP+FN},$$ (15)

$$\mathit{\text{Dice}}=\frac{2\times TP}{2\times TP+FP+FN}.$$ (16)

4.3. Results and Discussion

We constructed a loss function (11) that enables the optimization of CNN models to segment a highly imbalanced foreground using the Dice term, besides encoding the prior knowledge of the topological characteristics of the segmentation results into the dense layer classifier through the topological loss term.

We compared the performance of our proposed model to the-state-of-the-art baselines quantitatively in Table 1 and qualitatively in Figure 4. Our model outperforms other models in terms of Dice and Jaccard indices. Since foreground voxels are less than 10% of the population, these two metrics are better indicators of the actual performance of a model in comparison to other metrics such as accuracy. A classifier that classifies all the input voxels as backgrounds results in 90% accuracy and 100% specificity (or 100% sensitivity if it classifies all the voxels as foregrounds); nevertheless, its Dice and Jaccard indices are 0%. Under other models, the results will have slightly higher accuracy, sensitivity, and specificity; but with lower Dice and Jaccard indices in addition to having topological errors (Figure 4). However, these shortcomings demonstrate the existing capacity for improvement, mainly because this tool is frequently used in different branches of biomedical research and also the dependency of downstream image analysis tasks, such as the centerline extraction, on the quality of the segmentation results.

Table 1.

Result of our model in comparison to the baseline models. Baseline models are VIDA [42], VesselNN [41], U-Net [9], UMIS [19], and DeepVess [21]. The measured performance metrics are accuracy, sensitivity, specificity, Dice coefficient, and Jaccard index, with the last two metrics being correlated. Our model presents a higher Dice and Jaccard indices while maintaining comparable performance in terms of the other metrics.

Model	Accuracy	Sensitivity	Specificity	Dice	Jaccard
VIDA [42, 19]	60.2%	99.8%	56.4%	30.4%	17.9%
VesselNN [41, 21]	95.1%	62.4%	98.7%	69.7%	55.1%
3D U-Net [9, 22]	95.6%	70.0%	98.2%	72.7%	59.4%
UMIS [19]	98.6%	99.2%	98.6%	82.9%	70.8%
DeepVess [21]	96.4%	90.0%	97.0%	81.6%	69.2%
Ours	96.8%	96.4%	98.2%	86.5%	76.2%

Figure 4.

Qualitative comparison between our model and DeepVess [21]. A-B. The cross-sections spaced through the depth are plotted. The first column is the image intensity of the 2D slice in gray-scale. The second and third columns are the results of DeepVess and our models overlaid on the intensity images. The last column of each panel is the entropy with cooler and warmer colors representing the lower and higher uncertainty levels, respectively. C. The results of DeepVess and our models applied to the last sample. The cyan and yellow arrows represent the discontinuity and hole artifacts, respectively. Scale bar is 100μm.

The usage of infrared light to excite fluorophores with high absorption cross-sections enables deeper imaging in light-scattering tissues with low energy requirements and high resolution. However, the main sources of image processing difficulties are the rapid loss of signal and the background signal from the superficial layer in the images acquired deeper within the tissue. Due to this intrinsic limitation, the study of the model performance as a function of depth is essential. Figure 3 is the box-plot of the Dice Coefficient measured at each slice for all slices in the 3D image. Our model has lower Dice variation in comparison to DeepVess, while our model maintains a similar median value for the performance, which is an indicator that our model is less sensitive to the SNR variation over the depth of imaging with higher consistency.

Figure 3.

2D slice-based Dice coefficient variation as a function of imaging depth based on our model and baselines. On each box, the central red line indicates the median, while the bottom and top limits of the box indicate the first and third quartiles, respectively. The whiskers do not include outliers, which are plotted using the plus sign.

Figure 4 illustrates the results of our model and DeepVess overlaid on the input images. In Figure 4.A–B, the cross-sections spaced through the depth are plotted. The first column is the image intensity of the 2D slice in grayscale. The second and third columns are the results of DeepVess and our model overlaid on the intensity images. The input images were divided into the input patches using different splitting schemes to measure our model uncertainty by measuring Shannon’s entropy. Higher entropy values correlate with higher segmentation uncertainties. The last column is the entropy with cooler and warmer colors representing the lower and higher uncertainty levels, respectively.

Although the deeper slices, e.g. Figure 4.B, have more voxels with uncertainty; evidently, the level of uncertainty is lower than the results in [21] and the topological features of the segmentation results are close to the ground truth and the topological priors in our application. In Figure 4.C, the cyan and yellow arrows in DeepVess’ result represent the discontinuity and hole artifacts, respectively. The segmentation results based on our model do not have any hole in addition to the fact that the integrity and continuity of the vessel are very well-maintained. These two characteristics are essential for the downstream image analysis tasks, including the vessel centerline extraction.

5. Conclusion

We proposed a convolutional neural network architecture with a loss function consists of three terms, including a topological loss term. The topological loss term is based on the persistent homology and encodes the prior knowledge of topological features and their characteristics into the model. We demonstrated the application of this model to the segmentation of brain vasculature networks in multiphoton microscopy images in mice. While our model outperforms the-state-of-the-art models based on the Dice and Jaccard indices, our model has comparable accuracy, sensitivity, and specificity in addition to producing segmentation results with less topological errors in comparison to the-state-of-the-art models.