Shape-Aware 3D Small Vessel Segmentation with Local Contrast Guided Attention

Abstract

The automated segmentation and analysis of small vessels from in vivo imaging data is an important task for many clinical applications. While current filtering and learning methods have achieved good performance on the segmentation of large vessels, they are sub-optimal for small vessel detection due to their apparent geometric irregularity and weak contrast given the relatively limited resolution of existing imaging techniques. In addition, for supervised learning approaches, the acquisition of accurate pixel-wise annotations in these small vascular regions heavily relies on skilled experts. In this work, we propose a novel self-supervised network to tackle these challenges and improve the detection of small vessels from 3D imaging data. First, our network maximizes a novel shape-aware flux-based measure to enhance the estimation of small vasculature with non-circular and irregular appearances. Then, we develop novel local contrast guided attention(LCA) and enhancement(LCE) modules to boost the vesselness responses of vascular regions of low contrast. In our experiments, we compare with four filtering-based methods and a state-of-the-art self-supervised deep learning method in multiple 3D datasets to demonstrate that our method achieves significant improvement in all datasets. Further analysis and ablation studies have also been performed to assess the contributions of various modules to the improved performance in 3D small vessel segmentation. Our code is available at https://github.com/dengchihwei/LCNetVesselSeg.

1. Introduction

The automated detection and analysis of small vessels from non-invasive imaging data is critical for many clinical studies such as the research on cerebral small vessel disease(CSVD) [3], which is the most common vascular cause of dementia in Alzheimer’s disease and related dementia (ADRD) [5]. According to [12], a brain vessel with a diameter less than 0.5mm is considered as a small vessel by most pathologists. Fortunately, with the recent advances in MRA at 7-Tesla 7 and black-blood MRI at 3- and 7-Tesla [10], it is now possible to detect the small cerebral vessels directly. Although the segmentation of large vascular structures has been well studied for many years [4, 6, 8, 11, 15], accurate and reliable small vessel segmentation remains a challenging task. In contrast to large vessels, small vessels usually exhibit the following two main characteristics(Fig. 1). (1) Its cross-section only occupies a few image voxels due to the limited resolution of imaging techniques such as the magnetic resonance angiography(MRA). This makes the regular assumption of tube-like shape for vessels often does not hold. (2) Small vessels often have weak intensities and low contrasts, which would be easily affected by noise or surrounding backgrounds. These characteristics are generally not well modeled by existing methods for vessel detection [4, 6, 8, 11, 15].

Fig. 1. Challenges in small vessel detection.

(a) A high-resolution MRA image patch (0.2 × 0.2 × 0.4 mm) acquired by a 7T Simens Terra scanner. (b) A black-blood MRI image patch (0.5 × 0.5 × 0.5 mm) acquired by a Siemens 3T Prisma scanner. In each case, a maximal(minimal) intensity projection (left) and two cross sections of small vessels (right) were plotted. In (a) and (b), each cross-section corresponds to the line of the same color overlaid on the left panel (green: irregular appearances of small vessel cross-sections, red: low contrasts of small vessel cross-sections).

Traditional vesselness filters typically characterize blood vessels based on hand-crafted features [4, 8, 15]. The inherent assumption about the regularity of tube-like vessel geometry, however, makes them sub-optimal for small vessel segmentation. In addition, most Hessian-based filters rely on complicated pre-processing including smoothing for the calculation of second derivatives, which can further weaken or even eliminate the contrasts of small vascular structures. While deep learning methods have been successfully applied in various segmentation tasks, large-scale annotated labels are hard to obtain to train supervised networks for the segmentation of highly variable small vessels. To overcome this challenge, a self-supervised deep learning approach was recently proposed that combines geometric models and deep neural networks to learn vessel flow directions [6], but it also assumes a circular tube-like vessel shape and is thus limited in segmenting arbitrary-shaped small vessel structures.

In this work, we propose a novel self-supervised network that focus on challenges in small vessel segmentation with the following contributions: (1) Instead of assuming ideal tube-shaped vessels like [6, 8], we propose an adaptive scheme for shape-aware estimation of oriented fluxes to model the irregular (non-circular) cross-section profiles of small vessels. (2) We propose the Local Contrast Attention (LCA) module based on a novel local contrast measure to enhance the small vessel pattern and suppress the background clutter simultaneously. (3) We propose a novel unsupervised learning framework that considers the characteristics of small vessel structures in relatively limited resolution for the first time. Comprehensive experiments show promising improvements in 3D datasets of multiple modalities compared with previous unsupervised approaches.

2. Method

Our proposed framework for small vessel detection is shown in Fig. 2, a U-shaped network augmented with multiple novel LCA and LCE modules to learn a general representation of irregular-shaped small vessels by optimizing a self-supervised loss of shape-aware flux. Formally, let $\mathrm{ }\mathrm{\Omega }\subseteq {\mathbb{R}}^3\mathrm{ }$ denote the image domain. To characterize the irregular shape of the small vessel at each point $\mathrm{ }x\in \mathrm{\Omega }$, we estimate a principal vessel direction $\mathrm{ }\overrightarrow{{\rho }_x}\mathrm{ }$ and a set of radii values $\mathrm{ }R\left(x\right)=\left\{r_i\left(x\right)\mid i=\mathrm{1,2},\dots ,m\right\}\mathrm{ }$ that represent the vessel radius along $\mathrm{ }m\mathrm{ }$ sampling directions on the unit sphere $\mathrm{ }{S}^2$. Based on the estimated $\mathrm{ }R\left(x\right)\mathrm{ }$ and $\mathrm{ }\overrightarrow{{\rho }_x}$, we can compute the vesselness score $\mathrm{ }{f}_{vs}\mathrm{ }$ at $\mathrm{ }x$, which represents the likelihood of $\mathrm{ }x\mathrm{ }$ being a vascular structure. The proposed network is trained to maximize the vessel score across $\mathrm{ }\mathrm{\Omega }$. Next we describe in detail the shape-aware flux and the local contrast guided attention modules to enhance the detection of small vessels.

Fig. 2. Network overview.

Our model jointly estimate the vessel directions and shapes using a vector field $\mathrm{ }P\mathrm{ }$ representing the principal direction of tubular structures and a collection of scalar fields $\mathrm{ }R\mathrm{ }$ denoting the radius of the vessel at different directions.

Shape-Aware Flux.

Given an input image $\mathrm{ }I$, our proposed network will generate three outputs: $\mathrm{ }P,R=\left\{r_1,r_2,\dots ,r_m\right\}\mathrm{ }$ and the reconstructed image $\mathrm{ }\hat{I}$. We denote $\mathrm{ }P\mathrm{ }$ as a vector field that represents the principal direction for every point $\mathrm{ }x\in \mathrm{\Omega }\mathrm{ }$ and $\mathrm{ }R\mathrm{ }$ as a collection of scalar fields, where $\mathrm{ }{r}_i\in R\mathrm{ }$ is a scalar field over $\mathrm{ }\mathrm{\Omega }\mathrm{ }$ that estimates the vessel radius along $\mathrm{ }{d}_i\in S^2(i=\mathrm{1,2},\cdots ,m)$. A projected flux response along a direction $\mathrm{ }\overrightarrow{\rho }$, which generalizes conventional flux measures for circular shaped tubes [8], can be defined at $\mathrm{ }x\mathrm{ }$ as follows:

$$f(x,R(x),\overrightarrow{\rho })=-\frac{1}{m}{\sum }_i \left(\left(v\left(x+\overrightarrow{h_i}\right)\cdot \overrightarrow{\rho }\right)\overrightarrow{\rho }\right)\cdot \overrightarrow{d_i}$$ (1)

after discretization and normalization, where $\mathrm{ }v(\cdot )\mathrm{ }$ is the image gradient, $\mathrm{ }\overrightarrow{h_i}=r_i\left(x\right)\overrightarrow{d_i}\mathrm{ }\mathrm{ }$ and $\mathrm{ }{r}_i\left(x\right)\in R\left(x\right)$. The vesselness score of $\mathrm{ }x\mathrm{ }$ can then be computed as

$$f_{vs}\left(x,R\left(x\right),\overrightarrow{{\rho }_1}\right)=-f\left(x,R\left(x\right),\overrightarrow{{\rho }_2}\right)-f\left(x,R\left(x\right),\overrightarrow{{\rho }_3}\right)$$ (2)

where $\mathrm{ }\overrightarrow{{\rho }_1}=P\left(x\right)\mathrm{ }$ is the estimated principal direction at $\mathrm{ }x\mathrm{ }$ and $\mathrm{ }\overrightarrow{{\rho }_2},\overrightarrow{{\rho }_3}\mathrm{ }$ are two orthogonal vectors in the cross-sectional plane of the vascular structure. In contrast to conventional flux measures for vessel segmentation, where only isotropic radius value is estimated, radius in different sampling directions $\mathrm{ }{d}_i\mathrm{ }$ are estimated adaptively in this work to fit the irregular-shaped vascular boundaries as $\mathrm{ }{f}_{vs}\mathrm{ }$ is maximized only if the sampling vectors $\mathrm{ }\overrightarrow{h_i}\mathrm{ }$ fit the vessel edges. Since the radius values can be different from each other, our proposed network is designed to handle the small vessels with irregular and non-circular cross-sections.

Local Contrast Guided Attention.

As shown in Fig. 1, geometric features of large vessels are well-presented due to their high signal-to-noise ratio(SNR). On the other hand, the low contrast of small vessel region makes it hard to distinguish the vascular structures from the background clutters. To address this issue, Fig. 3(a) illustrates our novel spatial attention module to enhance the small vessels based on regional contrast measure. In a vascular image, we assume the pixels inside a vessel have similar intensities even for small structures.

Fig. 3. Local contrast guided attention.

(a) shows the proposed LCA and LCE module framework; (b) is an axial slice of a black-blood MRI image that contains multiple low-constrast small vessels (pink box); (c) shows the learnt attention map for (b); (d-e) show the vesselness maps generated by our model without and with the proposed modules for small vascular regions, respectively.

Consider the estimated radius $\mathrm{ }R\left(x\right)\mathrm{ }$ for $\mathrm{ }x\in \mathrm{\Omega }$, we define a local contrast $\mathrm{ }D\mathrm{ }$ as

$$D(I,x,R(x),s)=\frac{1}{2}\sum _{i=1}^m \left|I\left(x\right)-I\left(x+s\times \overrightarrow{h_i}\right)\right|\left|I\left(x\right)-I\left(x+s\times \overrightarrow{h_j}\right)\right|$$ (3)

where $\mathrm{ }\overrightarrow{h_j}=r_j\left(x\right)\overrightarrow{d_j}\mathrm{ }$ and $\mathrm{ }\overrightarrow{d_j}=-\overrightarrow{d_i}$. With $\mathrm{ }s=1,D\mathrm{ }$ measures the intensity differences between $\mathrm{ }x\mathrm{ }$ and $\mathrm{ }x+\overrightarrow{h_i}\mathrm{ }$ which locates on vascular boundaries for every sampling direction $\mathrm{ }{d}_i$. We can contract or expand the vascular regions along its edges with different $\mathrm{ }s$. To measure the contrast inside and outside the vascular regions, we design two measures $\mathrm{ }{D}_{\mathit{\text{in}}}\mathrm{ }$ and $\mathrm{ }{D}_{\mathit{\text{out}}}\mathrm{ }$ as follows:

$$D_{\mathit{\text{in}}\mathrm{ }}(I,x)={\int }_0^1 D(I,x,R(x),s)ds,\mathrm{ }{D}_{\mathit{\text{out }}}(I,x)={\int }_1^2 D(I,x,R(x),s)ds$$ (4)

With $\mathrm{ }s\in \left[\mathrm{0,1}\right],D_{\mathit{\text{in}}}\mathrm{ }$ computes the intensity differences inside vessels. $\mathrm{ }{D}_{\mathit{\text{out}}}\mathrm{ }$ computes outside intensity differences similarly with $\mathrm{ }s\in \left[\mathrm{1,2}\right]$. As for the relations between $\mathrm{ }{D}_{\mathit{\text{in}}}\mathrm{ }$ and $\mathrm{ }{D}_{\mathit{\text{out}}}$, we consider 3 situations for different $\mathrm{ }x\in \mathrm{\Omega }$. (1) For $\mathrm{ }x\mathrm{ }$ inside vascular structures, $\mathrm{ }{D}_{\mathit{\text{in}}}<<D_{\mathit{\text{out}}}\mathrm{ }$ should stand since $\mathrm{ }\left|I\left(x\right)-I\left(x+s\times \overrightarrow{h_i}\right)\right|\approx 0\mathrm{ }$ when $\mathrm{ }s<1$; (2) $\mathrm{ }{D}_{\mathit{\text{in}}}\mathrm{ }$ and $\mathrm{ }{D}_{\mathit{\text{out}}}\mathrm{ }$ would give similar measures if $\mathrm{ }x\mathrm{ }$ is in the backgrounds, where we consider the local regional intensities are similar. (3) Since equation (3) is evaluated on two opposite directions, $\mathrm{ }{D}_{\mathit{\text{in}}}\mathrm{ }$ and $\mathrm{ }{D}_{\mathit{\text{out}}}\mathrm{ }$ will both give low measures if $\mathrm{ }x\mathrm{ }$ is an one-sided edge. Based on these three scenarios, we design a novel local contrast measure as

$$D_{LC}(I,x)=\mathit{\text{Sigmoid}}\left(\frac{D_{\mathit{\text{out }}}(I,x)}{D_{\mathit{\text{in }}}(I,x)+ϵ}-1\right)$$ (5)

where $\mathrm{ }ϵ\mathrm{ }$ is a small constant to prevent the numerical explosion. Since the ratio of $\mathrm{ }{D}_{\mathit{\text{out}}}\mathrm{ }$ and $\mathrm{ }{D}_{\mathit{\text{in}}}\mathrm{ }$ gives large value, $\mathrm{ }{D}_{LC}\approx 1\mathrm{ }$ for $\mathrm{ }x\mathrm{ }$ inside a vascular structure, which means the $\mathrm{ }{D}_{LC}\mathrm{ }$ measure of small vessels is on a similar scale to large vessels. For the latter two situations, $\mathrm{ }{D}_{LC}\approx \sigma \left(0\right)=0.5\mathrm{ }$ will stand since $\mathrm{ }{D}_{\mathit{\text{in}}}\approx D_{\mathit{\text{out}}}$. Therefore, the proposed $\mathrm{ }{D}_{LC}\mathrm{ }$ measure can enhance the small vascular structures and suppresses backgrounds and edges simultaneously.

Guided by the local contrast map of images, we propose to incorporate a novel spatial attention module in our DL network. Following CBAM [16], we use two pooling layers to abstract the previous features as two feature maps. A local contrast map is computed based on a coarsely estimated $\mathrm{ }\hat{R}$, which is generated by previous features. Then, all three maps are sent to a convolutional block to compute the final spatial attention map. By simply extending the LCA module, LCE module scales the previous features with the attention map to refine it. To include low-level contrast information, we insert our LCA and LCE module in the skip connections between encoders and decoders as shown in Fig. 2. As shown in Fig. 3(b–d), with the local contrast attention modules, our model can enhance the vesselness measure of the small vessels with low contrasts.

Self-supervised Losses.

Based on (2), we propose our flux-based loss $\mathrm{ }{L}_{\mathit{\text{flux}}}\mathrm{ }$ as the negative average vesselness score over the whole image spatial space. This cost function takes advantage of the clear edge separation of flux-based filter and robustness to irregular-shaped vessels due to our adaptive shape-aware flux computation. To ensure the vessel structure’s continuity, we adopt the path continuity loss $\mathrm{ }{L}_{\mathit{\text{path}}}\mathrm{ }$ from [6]. Let us denote $\mathrm{ }P\left(x\right)\mathrm{ }$ as the vascular principal direction at $\mathrm{ }x$, and $\mathrm{ }P(x+t)\mathrm{ }$ as the principal direction at the location $\mathrm{ }x+t$, which we obtain by walking for $\mathrm{ }t\mathrm{ }$ from $\mathrm{ }x\mathrm{ }$ along $\mathrm{ }P\left(x\right).L_{\text{path}}\mathrm{ }$ is designed to maximize the inner product of these two vectors to encourages $\mathrm{ }P\mathrm{ }$ to have a consistent and smooth direction along the vessels. Formally, $\mathrm{ }{L}_{\mathit{\text{flux}}}\mathrm{ }$ and $\mathrm{ }{L}_{\mathit{\text{path}}}\mathrm{ }$ are computed as

$$\begin{gathered} L_{\mathit{\text{flux }}}(P,R)=-\frac{1}{N}\sum _{x\in \mathrm{\Omega }} f_{vs}\left(x,R\left(x\right),\overrightarrow{{\rho }_x}\right) \\ {L}_{\mathit{\text{path }}}(P,R)=-{\int }_{\mathrm{\Omega }} {\int }_0^{2\stackrel{-}{R\left(x\right)}} \left(P\right(x)\cdot P(x+t\left)\right)\mathit{\text{dtdx}} \end{gathered}$$ (6)

where $\mathrm{ }N\mathrm{ }$ is the total number of voxels and $\mathrm{ }\stackrel{-}{R\left(x\right)}\mathrm{ }$ is the average radius of $\mathrm{ }R\left(x\right)\mathrm{ }$ so that the length of the walking path is relative to the vessel size at $\mathrm{ }x$. Mean square error loss is applied between the reconstructed image $\mathrm{ }\hat{I}\mathrm{ }$ and original image $\mathrm{ }I\mathrm{ }$ to make sure the network learns the semantic meaningful features based on the previous success of reconstruction-based self-supervised learning [9]. So, the overall objective of our network is expressed as

$$L={\lambda }_1{L}_{\mathit{\text{flux}}}+{\lambda }_2{L}_{\mathit{\text{path }}}+{\lambda }_{3}MSE(\hat{I},I)$$ (7)

where $\mathrm{ }{\lambda }_1,{\lambda }_2\mathrm{ }$ and $\mathrm{ }{\lambda }_3\mathrm{ }$ correspond to the coefficients of each objective.

3. Experimental Results

Datasets.

We used two public datasets and two in-house datasets in our experiments for 3D vessel segmentation. The VESSEL12 [14] contains 23 CT lung images and 3 of them are provided with sparsely annotated vessel and non-vessel locations along 3 axial slices, which we used as a test set. The TubeTk [2] consists of 109 3D brain MRA images. We used the 42 images with ground truths as the test set and the rest of the dataset for training. To better demonstrate the small vessel detection ability of our method, we collected a new brain MRA dataset on a 7T Simens Terra scanner from 31 subjects with voxel size of 0.2 × 0.2 × 0.4mm. For this dataset, we sparsely annotated 2200 small vessels(within 3 × 3 grids) locations on axial slices of 7 images according to the definition of the small vessel in [12] (Fig. 4(i–j)). The remaining 24 images constitute the training set. In addition, a black-blood MRI dataset was collected using a Siemens 3T Prisma scanner with voxel size of 0.5 × 0.5 × 0.5mm and separated by left and right hemispheres for a total of 56 image volumes. Dense manual segmentation of the lenticulostriate arteries(LSAs) was carefully performed by two experts. The dataset was divided into a training set with 21 subjects (42 volumes) and a test set with 7 subjects (14 volumes).

Fig. 4. Qualitative comparison.

(a-h) is a comparison for TubeTK dataset. Green: true positives; red: false negatives. (i-j) is an example of our manual label for a 7T MRA image patch; (k-l) is small vessel detection comparison for the patch in (i-j).

Implementation Details.

We performed the vessel segmentation tasks for each dataset to evaluate our model’s performance. For comparison, we selected 4 conventional filters, including the Frangi, Sato, Meijering and OOF filter and a flow-based DL method [6]. For fair comparison, the same pre-processing procedures were applied for all methods using FreeSurfer 6.0 [13] and the SimpleITK toolbox [1]. In addition, for the black-blood LSA dataset, the image intensities were inversed to make sure the vascular voxels are brighter than backgrounds.

To tune the network’s hyper-parameters, we used 15% of the training data as the validation set. Finally, we set $\mathrm{ }m=128,{\lambda }_1=5\mathrm{ }$ and $\mathrm{ }{\lambda }_2={\lambda }_3=1\mathrm{ }$ for our model. For both DL models, patch size is set to 64 × 64 × 64 and the networks are trained for 100 epochs. All experiments were conducted with Pytorch on one NVIDIA A5000 GPU with the Adam optimizer and a learning rate of 0.001.

For all methods, the output is a vessel enhanced image. The enhanced image is binarized through a hard threshold. We thus found the best threshold by optimizing the metrics based on the validation sets and then applied the final threshold to test sets. We reported five metrics in Table 1, namely, the area-under-curve(AUC), accuracy, sensitivity, specificity and dice score. For VESSEL12 and 7T MRA dataset, we treated the segmentation tasks as classification problems and dropped the dice score metric since they do not have dense labels.

Table 1.

Vessel Segmentation Performance Comparison on Different Datasets.

	VESSEL12				7T MRA
Method	Acc	Sens	Spec	AUC	Acc	Sens	Spec	AUC
Sato	0.8299	0.7580	0.8636	0.9102	0.8716	0.9705	0.7347	0.8933
Meijering	0.9184	0.8932	0.9301	0.9720	0.7060	0.9377	0.3854	0.6547
Frangi	0.9672	0.9584	0.9669	0.9738	0.8130	0.7008	0.9683	0.8104
OOF	0.9331	0.9324	0.9334	0.9659	0.8416	0.8074	0.8889	0.8623
Flow-Based	0.9553	0.9856	0.9213	0.9871	0.9247	0.9432	0.9261	0.9485
Ours	0.9761	0.9809	0.9793	0.9937	0.9824	0.9787	0.9875	0.9972
	TubeTK				Black-blood LSA
Method	Dice	Sens	Spec	AUC	Dice	Sens	Spec	AUC
Sato	0.3166	0.3933	0.9964	0.9262	0.0596	0.7111	0.92251	0.9432
Meijering	0.1573	0.1604	0.9970	0.9023	0.1669	0.6512	0.9797	0.9459
Frangi	0.3569	0.3641	0.9977	0.9319	0.2667	0.6221	0.9905	0.9281
OOF	0.3877	0.3874	0.9980	0.9426	0.3368	0.6454	0.9886	0.9584
Flow-Based	0.4003	0.4211	0.9978	0.9693	0.4608	0.6673	0.9927	0.9732
Ours	0.5487	0.5061	0.9987	0.9878	0.5121	0.6979	0.9983	0.9624

Results and Discussion.

We can clearly observe from Table 1 that our model outperforms all other methods by a significant margin for all datasets. Since the flow-based DL outperformed the conventional filters, we will use it for visual comparisons with our method. From Fig. 4(a–d), we can observe that our model generates more accurate segmentation with more true positives and less false negatives as compared to the flow-based DL method [6]. These results show that our model can better detect general vessel structures including large vessels.

Small Vessel Segmentation.

From Fig. 4(e–h), we can see the main improvement of our model over the flow-based DL method occurs often at areas of small vessels, where our results have much lower false negatives for small vessels missed by the other. The results on the 7T MRA data in Table 1 further demonstrate that our model achieved better performance on the small vessel detection task. Fig. 4(k–l) provides a visual comparison of the performance of small vessel detection by our method and the flow-based DL, which shows that our model can more successfully segment the vessels with very small radius and low contrasts. In addition, Fig. 5 shows that our estimated principal directions are better aligned with the vessel directions over the flow-based DL. Furthermore, our estimated radius can better capture the vascular shapes by fitting the asymmetric vessel boundaries. Thus, comparing with the flow-based DL, our model can produce sharper and clearer vesselness maps(Fig. 5).

Fig. 5. Vessel Estimation Comparison.

Comparison between our model and flow-based model on vessel direction and shape estimations.

Ablation Study.

To investigate the contributions of each proposed module in our approach, we performed ablation studies by training the network with each proposed module using the 7T MRA dataset. The results are shown in Table 2, where we used the flow-based DL method as the baseline for comparison. Note that CS stands for circular sampling and AS stands for the proposed shape-aware adaptive sampling. As can be seen, with all the proposed modules, we increase the AUC metric of the baseline by 4.87% on the test set, showing significant improvements with only minimal extra computational cost. Furthermore, we observed that all the proposed modules improve both the sensitivity and specificity since complicated small vessels can be better modeled by these components.

Table 2.

Ablation Analysis Results with the 7T high-resolution MRA Dataset.

Methods	Acc	Sens	Spec	AUC	# of Params
Flow-based [6]	0.9247	0.9432	0.9261	0.9485	32.63M
Ours (CS only)	0.9344	0.9478	0.9329	0.9554	32.63M
Ours (AS only)	0.9578	0.9669	0.9333	0.9662	32.64M
Ours (LCA only)	0.9747	0.9699	0.9732	0.9865	32.65M
Ours (AS + LCA)	0.9824	0.9787	0.9875	0.9972	32.66M

4. Conclusions

In this paper, we proposed a self-supervised network for the detection of small vessels from 3D imaging data. Our method is designed to address existing challenges in small vessel detection arising from their irregular appearance in relatively limited resolution and low-contrast conditions. In comparison to previous methods, we demonstrated that our method is able to achieve superior performance on small vessel detection. For future work, we will also apply it to various clinical datasets to examine its power for CSVD detection in brain images.