Human microscopic vagus nerve anatomy using deep learning on 3D-MUSE images

Abstract

We are microscopically imaging and analyzing the human vagus nerve (VN) anatomy to create the first ever VN connectome to support modeling of neuromodulation therapies. Although micro-CT and MRI roughly identify vagus nerve anatomy, they lack the spatial resolution required to identify small fascicle splitting and merging, and perineurium boundaries. We developed 3D serial block-face Microscopy with Ultraviolet Surface Excitation (3D-MUSE), with 0.9-μm in-plane resolution and 3-μm cut thickness. 3D-MUSE is ideal for VN imaging, capturing large myelinated fibers, connective sheaths, fascicle dynamics, and nerve bundle tractography. Each 3-mm 3D-MUSE ROI generates ~1,000 grayscale images, necessitating automatic segmentation as over 50-hrs were spent manually annotating fascicles, perineurium, and epineurium in every 20th image, giving 50 images. We trained three types of multi-class deep learning segmentation models. First, 25 annotated images trained a 2D U-Net and Attention U-Net. Second, we trained a Vision Transformer (ViT) using self-supervised learning with 200 unlabeled images before refining the ViT’s initialized weights of a U-Net Transformer with 25 training images and labels. Third, we created pseudo-3D images by concatenating each annotated image with an image ±k slices apart (k=1,10), and trained a 2D U-Net similarly. All models were tested on 25 held-out images and evaluated using Dice. While all trained models performed comparably, the 2D U-Net model trained on pseudo-3D images demonstrated highest Dice values (0.936). With sample-based-training, one obtains very promising results on thousands of images in terms of segmentation and nerve fiber tractography estimation. Additional training from more samples could obtain excellent results.

1. INTRODUCTION

Vagus Nerve Stimulation (VNS) is a clinically approved therapy used to treat patients suffering from drug-resistant epilepsy¹, depression², and obesity³. However, VNS is limited in targeting and therapeutic efficacy causing failure to reach clinical endpoints and inducing negative side-consequences^4-6. Computational modeling can aid the optimization of various simulation parameters and cuff designs, but accurate information about nerve fiber organization and biomarkers can improve fiber selectivity and simulations⁷. For example, perineurium thickness and fascicle area are key contributors to the neuromodulation simulations of VNS. We previously employed MUSE, Microscopy with Ultraviolet Surface Excitation, imaging to visualize peripheral nerve morphology⁸. MUSE imaging, unlike micro-CT and MRI, is able to identify small (sub-100-um) fascicle splitting and merging, and perineurium boundaries. Furthermore, we developed, a Python-based software for tracking fibers from a nerve sample ROI’s serial block-face microscopy images. The software estimates local fiber orientation from seed points, initialized via a fascicle mask image of the first sample slice, and through each image thereafter generating a tractogram representing fiber tracts through the ROI. To constrain the resulting tractogram with anatomical boundaries and determine important anatomical biomarkers, we investigated various deep learning approaches and developed a self-labeled-training multi-class segmentation model to identify the fascicle, perineurium, and epineurium within a vagus nerve MUSE imaging sample.

2. METHODOLOGY

2.1. Dataset

This work imaged and analyzed a 3-mm human vagus nerve ROI from the mid-cervical section of a human cadaver vagus nerve. We acquired 1000 grayscale images from this 3-mm ROI sample. Each 2D grayscale image tile was [3000, 4000] with 0.9-μm in-plane pixel size and 3-μm cut thickness. For training and testing of the multi-class segmentation model, 50 evenly spaced images were selected from the nerve ROI for manual annotation. The fascicles were annotated as class 1, perineurium—class 2, epineurium—class 3, and background—class 0. Starting from the first annotated image, every other annotated image, 25 images in total, was used for training of the multi-class segmentation model, and the remaining 25 images were in the held-out test set. Prior to inputting any training or testing image into a deep learning model, they were down-sampled to [768, 1024] to maintain whole image training.

2.2. Network Architectures

This work employed three main networks for deep learning multi-class segmentation: U-Net⁹, Attention U-Net¹⁰, and U-Net Transformer (UNETR)¹¹. U-Net uses skip connections to recover full spatial resolution in its decoding layers, allowing one to train such deep fully convolutional networks for semantic segmentation. Attention U-Net integrates attention gates into its architecture, which enable the network to focus on relevant structures within the image that may vary in shape and size while ignoring irrelevant regions. Finally, U-Net Transformers have shown promising results in the medical imaging segmentation field, as they are able to capture global multi-scale information by patching input images and sequentially arranging the input patches for model training.

2.3. Training Experiments

We conducted three deep learning experiments performing multi-class segmentation MONAI (Medical Open Network for AI) Python framework. First, we trained 2D models using a U-Net⁹ and Attention U-Net¹⁰ architecture on 25 annotated images for multiclass segmentation before testing on 25 held-out images. The second deep learning experiment employed a self-learning technique¹² where a U-Net Transformer (UNETR) autoencoder, a modified vision transformer (ViT)¹³, was pre-trained on 200 unlabeled images from the sample ROI. In this step, the model learns to recreate small, masked areas of the input images from low-dimensional representations. With this approach, the model learns inherent structures and features from the input data that can be utilized in a downstream task such as segmentation. The ViT’s pre-trained weights were then loaded onto a UNETR, which was then refined using the sample ROI’s 25 training images and labels before testing on its 25-image test set. Third, we created pseudo-3D images¹⁴ by concatenating each of the 25 training images with an image prior to and after each labeled image forming a 3-channel image. We also created a second set of pseudo-3D images by concatenating each labeled image with an image 30-μm (10 slices) prior and after, forming a new 3-channel image. The 25 pseudo-3D images (3-channel 2D images) and corresponding 1-channel label trained a 2D U-Net multiclass segmentation model which later inferenced on the held-out 25 psuedo-3D images.

2.4. Performance Assessment

We assessed our model performances by calculating the Dice Coefficient within each class (fascicle, perineurium, and epineurium) and by taking an average of the three class Dice Coefficients. Furthermore, we applied the best model to all 1,000 grayscale images of the nerve ROI and visualized a 3D rendering of each segmented class.

3. RESULTS

The mid-cervical human vagus nerve sample underwent inference following all three training experiments. The 2D U-Net Attention U-Net, and the self-supervised pre-trained UNETR model performed comparably between and across all three classes with 0.933, 0.929 and 0.925 average Dice coefficients from the 25 test images, respectively. The pseudo-3D images with k ± 1 and k ± 10 slices slightly outperformed the previous 2D networks generating average Dice coefficients of 0.935 and 0.936, respectively. Table 1 below details each class’s Dice score following each model’s inference on the held-out test set. Figure 1 displays predicted segmentations in comparison to manual annotations of example images within the sample ROI. Figure 2 visualizes 3D renderings of the sample ROI class segmentation predictions. Fascicle splitting and merging are evident within the 3-mm sample. The tortuosity of smaller fascicles within the sample is also clearly visible.

Table 1.

Dice Coefficient of various models and network architectures.

Model Network Architecture	Dice Coefficient
	Fascicle	Perineurium	Epineurium	Average
2D U-Net	0.965	0.876	0.960	0.933
2D Attentino U-Net	0.962	0.871	0.953	0.929
2D SSL UNETR	0.958	0.857	0.958	0.925
Psuedo-3D images 2D-UNet (k±1)	0.965	0.878	0.961	0.935
Psuedo-3D images w/ 2D-UNet (k±10)	0.965	0.880	0.963	0.936

Figure 1.

Multi-class segmentation predictions of three example 3D-MUSE images (a-c) from a mid-cervical human cadaver vagus nerve including corresponding ground truth manual annotations. Fascicles (green), perineurium (yellow), and epineurium (red)

Figure 2.

3D rendering of MUSE ROI segmentation predicted by 2D U-Net trained on pseudo-3D images 30-μm apart. Multi-class segmentation prediction of corresponding 3D-MUSE images identifying the fascicles (green), perineurium (yellow), and epineurium (red)

4. CONCLUSION

This work demonstrates multi-class segmentation utilizing various deep learning and pre-training techniques. While the 2D-UNet model trained on pseudo-3D images spaced 1 or 10 slices apart demonstrated the highest Dice, the 2D models with self-supervised learning and UNETR architecture, classic U-NET, and Attention U-Net also provide comparable results. Most likely the pseudo-3D images spaced with 10 slices performed best due to visual differences between the input images within the pseudo-3D stack as compared to images spaced only 1 slice apart. We will continue to investigate a global model trained on more samples or enhancing pseudo-3D images to include more slices, the advantage of large datasets could improve model performance.