Multi-Modal and Targeted Imaging Improves Automated Mid-Brain Segmentation

Abstract

The basal ganglia and limbic system, particularly the thalamus, putamen, internal and external globus pallidus, substantia nigra, and sub-thalamic nucleus, comprise a clinically relevant signal network for Parkinson's disease. In order to manually trace these structures, a combination of high-resolution and specialized sequences at 7T are used, but it is not feasible to scan clinical patients in those scanners. Targeted imaging sequences at 3T such as F-GATIR, and other optimized inversion recovery sequences, have been presented which enhance contrast in a select group of these structures. In this work, we show that a series of atlases generated at 7T can be used to accurately segment these structures at 3T using a combination of standard and optimized imaging sequences, though no one approach provided the best result across all structures. In the thalamus and putamen, a median Dice coefficient over 0.88 and a mean surface distance less than 1.0mm was achieved using a combination of T1 and an optimized inversion recovery imaging sequences. In the internal and external globus pallidus a Dice over 0.75 and a mean surface distance less than 1.2mm was achieved using a combination of T1 and F-GATIR imaging sequences. In the substantia nigra and sub-thalamic nucleus a Dice coefficient of over 0.6 and a mean surface distance of less than 1.0mm was achieved using the optimized inversion recovery imaging sequence. On average, using T1 and optimized inversion recovery together produced significantly improved segmentation results than any individual modality (p<0.05 wilcox sign-rank test).

1. Introduction

The basal ganglia and limbic system, consisting of structures like the thalamus, putamen, internal and external globus pallidus, substantia nigra, and subthalamic nucleus, are important relay centers in the human brain. These structures have been implicated in movement and movement disorders like Parkinson's disease [1], memory diseases like Alzheimer's [2], and cognition [3]. Specifically, the internal globus pallidus and sub-thalamic nucleus are targets for deep brain stimulation surgery [4, 5]. Thus, accurate, automated segmentation of these structures is desirable to localize these targets and assist in the surgical process.

The basal ganglia is involved in cognitive processing, motor control, and procedural learning [6-8]. The striatum receives glutamatergic and dopaminergic inputs and serves as the primary input to the basal ganglia [9]. The external globus pallidus (GPE) receives dopaminergic signal from the subthalamic nucleus (STN) and has signaling neurons projecting to other parts of the basal ganglia. The internal globus pallidus (GPI) is an output region of the basal ganglia, receiving signals from the subthalamic nucleus and sending signals to the thalamus [10, 11]. The STN is a transmission nucleus in the basal ganglia, involved in action selection and reward control [4]. The substantia nigra is the second output, sending signals from the striatum to various parts of the brain [12]. Deep brain stimulation surgery is a treatment option in Parkinson's disease. In deep brain simulation surgery, a neurosurgeon implants bi-lateral electrodes into the patient's brain. In Parkinson's the typical anatomic targets are the GPI and STN, making proper localization on imaging an important goal to assist with surgical planning. After completion of the surgery, the electrodes are activated to stimulate neural activity in the motor tracts and alleviate the motor symptoms of Prakinson's, though the surgery does not always restore normal motor function.

Imaging of these structures in the mid-brain typically requires high-field imaging and specialized sequences [13, 14]. For instance, susceptibility-weighted images are typically used to visualize the sub-thalamic nucleus and substantia nigra (SN). These sequences are typically slow and lose much of their contrast when done at clinically viable field strengths. Furthermore, the internal and external globus pallidus are separated by a thin lamina which is typically 2mm in diameter and has low contrast with standard imaging sequences [14, 15].

The Fast Gray Matter Acquisition T1 Inversion Recovery (F-GATIR) family of sequences has been optimized for contrast in the mid-brain structures [16]. To date, these sequences have not been evaluated for multi-atlas segmentation of the mid-brain structures, in particular in combination with other standard imaging. Further, algorithms for optimized segmentation when no one imaging modality is best for multi-atlas segmentation of the structures considered. In this work we present an algorithm for segmentation of six deep brain structures, the thalamus, putamen, internal and GPE, SN, and the STN, where the optimal imaging modality, or group of modalities, for segmentation of each structure may vary between structures. This algorithm focuses on localized segmentation of each structure, where the registration of each atlas to the target is localized to the specific region of the anatomy, thus allowing for different groups of modalities to be used for each independent segmentation problem.

2. Methods

For a series of 9 healthy subjects, each subject was scanned at 3T and 7T. Each subject's left thalamus, putamen, internal and external globus pallidus, SN, and STN were manually traced. The 9 subjects were then segmented in a leave-one-out cross-validation using the standard T1-weighted scan, an optimized inversion recovery scan, Fractional Anisotropy (FA), and multi-modally with the T1 and an optimized inversion recovery scan. Overlap and surface distance measures were then calculated to assess significant improvements between the segmentation approaches.

Imaging Sequences

For each subject, the following imaging sequences were acquired on each of the 9 healthy subjects. At 7T, a series of 0.7mm isotropic T1-weighted MP-RAGE (TI/TR/TE=[400,640,960,1120]/4.74/2.09ms) was acquired and a susceptibility weighted image slab through the midbrain acquired at 0.2×0.2×1.1mm was acquired in sagitally, coronally, and axially (TR/TE/FA=1952/23ms/45° for all orientations). At 3T, a 1.0mm isotropic resolution T1-weighted MP-RAGE (TI/TR/TE=925/8.1/2.7ms) and 2.0mm isotropic resolution 70 direction high-angular resolution diffusion weighted image (HARDI) (TR/TE=14500/64.5ms). An inversion recovery scan (O-IR), similar to an F-GATIR [16], was also optimized at 3T for mid-brain contrast. Inversion times were scanned from 200ms to 900ms in intervals of 100ms and the scan with the highest contrast between the thalamus and surrounding tissue was selected and scanned for each subject (TI/TR/TE=400/7.39/3.43).

Manual Segmentation

For each subject, the 7T T1-weighted MP-RAGE with the inversion time of 960ms was used as the reference space. The other 7T MP-RAGE scans, 3T MP-RAGE, 7T high-resolution susceptibility weighted slabs, and 3T OIR were co-registered to the reference space. Lastly, fractional anisotropy (FA) was calculated from the HARDI scan and was also co-registered to the reference space. The following structures were manually traced on each reference subject: thalamus, putamen, GPI, GPE, SN, and STN (Figure 1).

Figure 1.

Examples of five structures considered in this study shown on each of the imaging sequences considered in this study. The standard T1-weighted image shows high contrast for the putamen, but less contrast in the other structures considered. The OIR sequence shows improved resolution between the GPI and GPE and the SN, STN and surrounding tissue. Though FA would have contrast between the internal structures of the putamen/GPI/GPE complex, the 2.0mm isotropic resolution makes visualization of the lamina separating these structures not visible.

Segmentation Pipeline

First, each subject, the 3T T1-weighted MRI was automatically segmented with the BrainCOLOR (www.neuromorphometrics.com) following a standard multi-atlas segmentation approach [17-20]. This whole-brain segmentation (WBS) was then used to localize the thalamus from the thalamus label, putamen, GPI, and GPE from the putamen and globus pallidus labels, and the substantia nigra and sub-thalamic nucleus from the diencephalon label. The T1, OIR, and FA images around each of these structures were isolated with a 5mm bounding box around the structure to ensure the entire structure and enough contextual information was included. This results in a collection of reduced field of view (RFOV) atlases.

For a given target, the target was segmented with the BrainCOLOR protocol (www.neuromorphometrics.com). A series of targets (RFOV)were created following the protocol defined above. The RFOV atlases were co-registered to the RFOV targets. All registrations were performed using the Advanced Normalization Tools (ANTs) and the Symmetric Normalization (SyN) algorithm [21]. After registration, joint label fusion (JLF) was used. In all cases, the same collection of imaging modalities was used for the segmentation [22]. After each structure's segmentation it was reinserted into the standard image space.

Statistical Analysis

The nine subjects were segmented following the above pipeline in leave-one-out cross-validation using the procedure described above. Four different registration and segmentation schemes were used: FA, T1, OIR, and T1+OIR. This procedure results in four distinct segmentations for each subject. For each automated segmentation, the mean surface distance and Dice coefficient were calculated between the manual segmentation and the automated segmentation. Wilcox sign-rank tests were used to assess statistical significance between segmentation approaches.

3. Results

On average, the combination of T1+OIR provided the best results in average dice and mean surface distance across all the structures considered (p<0.05, Wilcoxon sign-rank test; Figure 2). FA produced significantly worse results than any other modality and OIR outperformed T1 alone. For the internal globus pallidus the combination of T1+OIR outperformed all other modalities in both Dice and mean surface distance (p<0.05, Wilcoxon sign-rank test; Figure 3). In the external globus pallidus T1 and T1+OIR outperformed other modalities in Dice and T1+OIR outperformed all other modalities in mean surface distance (p<0.05, Wilcoxon sign-rank test; Figure 3). In the substantia nigra, OIR outperformed all other modalities in Dice and OIR, T1, and T1+OIR outperformed FA, but were not significantly distinguishable from each other (p<0.05, Wilcoxon sign-rank test; Figure 3). In the sub-thalamic nucleus, OIR and T1+OIR significantly outperformed other modalities in both Dice and mean surface distance (p<0.05, Wilcoxon sign-rank test; Figure 3). In the putamen, OIR, T1, and T1+OIR outperformed FA in Dice and mean surface distance (p<0.05, Wilcoxon sign-rank test; Figure 3) but were not significantly differentiable from each other. In the thalamus OIR, T1, and T1+OIR outperformed FA in Dice and T1+OIR outperformed all other modalities in mean surface distance (p<0.05, Wilcoxon sign-rank test; Figure 3).

Figure 2.

Dice similarity coefficient and mean surface distance results for the nine subjects, averaged over each of the six labels. T1+OIR show a significant improvement in Dice and mean surface distance. FA produced significantly worse results than all other modalities considered. The OIR sequence outperformed T1 in both Dice and mean surface distance.

Figure 3.

Dice similarity coefficient and mean surface distance results for the nine subjects, for each the six labels. There was no consensus modality which outperformed all other modalities. In general, T1+OIR performed either best or not significantly worse than the top performing algorithm (*, p<0.05, Wilcoxon sign-rank test).

4. Discussion

In this work, we presented an algorithm for segmentation of six mid-brain structures. The manual delineation of these structures is typically done using 7T imaging. It is not often feasible to image all patients and subjects at 7T, so algorithms that segment these structures using imaging at 3T is desirable. The work presented uses segmentation manually delineated at 7T and transferred to subjects at 3T with both standard imaging, such as T1-weighted MP-RAGE and HARDI imaging sequences, and an optimized inversion recovery sequence for mid-brain contrast. This optimized sequence was similar to the F-GATIR imaging sequence, but has improved contrast for delineation of the internal and external globus pallidus. We showed that the combination of imaging sequences provided outperforms any individual sequence on the average, but at times individual sequences outperform multi-modal segmentation. Qualitatively, the segmentations produced by FA had more rigid edges because the FA images were acquired at a higher resolution. The segmentations produced by T1 and OIR tended to be more consistent with the true boundaries of structures, since T1-weighting produces high contrast for these structures. The combination of T1 and OIR produced significantly improved results in cases where the imaging modalities produced complimentary signal for the T1 and OIR. For instance, the T1-weighted MPRAGE provides better thalamus/cerebrospinal fluid contrast whereas OIR provides better thalamus/white matter contrast (Figure 4). Using a reduced-field-of-view approach allows us to select which combination of sequences is ideal for a given target structure.

Figure 4.

Qualitative segmentation results for a select set of regions where the red corresponds to the manual segmentation and the green corresponds to the column's segmentation modalities. For all segmentations, FA produced jagged boundaries which did not well correspond with the true segmentation. One case where T1+OIR outperformed the individual modalities was in the thalamus. T1 produced better results between the thalamus and cerebrospinal fluid whereas the OIR better captures the boundary between the thalamus and white matter. T1+OIR captured both of these boundaries better than the individual modalities.