Machine learning for white matter fibre tract visualization in the human brain via Mueller matrix polarimetric data

Abstract

A clear identification of the border between a brain tumor and surrounding healthy tissue during neurosurgery is essential in order to maximize tumor resection while preserving neurological function. However, tumor tissue is often difficult to differentiate from infiltrated brain during surgery. Most existing techniques have drawbacks in terms of cost, measurement time and accuracy. The fibre tracts of healthy brain white matter are composed of densely packed bundles of myelinated axons that form uniaxial linear birefringent medium with the optical axis oriented along the direction of the fibre bundle. Brain tumors, whose cells grow in a largely chaotic way, lack this anisotropy of refractive index. Therefore tumor tissue can be distinguished from of healthy white matter using polarized light. A wide-field visible wavelength imaging Mueller polarimetric system was used for the study of formalin-fixed human brain sections measured in reflection geometry. The non-linear decomposition of the Mueller matrices provided the maps of depolarization, scalar retardance and azimuth of the optical axis. A compelling correlation between the azimuth of the optical axis and the orientation of the brain fibre tracts was proven with the gold standard histology analysis. We present the results of post-processing of Mueller polarimetric images of fixed human brain sections using a combination of classical computer vision and machine learning algorithms, for the automated brain fibre tracking in the white matter tracts. Manually labelled polarimetric data was used to train a convolutional neural network to identify white matter. Within the identified white matter, surface fibre tracts could be visualized. We expect that Mueller polarimetric imaging modality combined with our ML algorithms for fibre tracking will visualize the directions of fibre tracts in imaging plane during tumor surgery, thus, allowing a neurosurgeon to orient himself, to spare essential fibre tracts and to make surgery more complete and safe.

1. Introduction

Surgery is the first-line treatment for most brain tumor cases.^{1, 2} While tumor tissue can be readily distinguished from the surrounding tissue on magnetic resonance imaging, during surgery the two tissues have very similar appearance. The goal of the surgeon is to remove as much tumor tissue as possible, including a wide safety margin if possible, without disturbing vital white-matter tracts which are essential to neurological function. For this reason, it is not only crucial to visualize the border between tumor and healthy tissue, but also to determine the directions of white-matter tracts appearing in the resection cavity. These can help the neurosurgeon to distinguish highly eloquent tissue near the tumor margin from tissue which may be safely removed.

We have previously noted the potential of wide-field imaging Mueller polarimetry operating in reflection geometry to visualize white matter fibre tracts in both fixed human brain specimens and fresh calf brains.^3–5 In this work we extend this approach to the visualization of white-matter fibre tracts in fixed human brain sections by applying machine learning algorithms. In addition to white matter and grey matter, ex vivo polarimetric images of these sections contain non-interesting background elements. Since we do not expect to see biologically meaningful surface fibre tracts inside the grey matter at the scale of hundred of microns (the spatial resolution of our wide-field imaging Mueller polarimeter), it is of interest and practically necessary to separate the white matter pixels from other pixels in the image of brain tissue. In this paper we explore the separation of these classes using standard polarimetric parameters (total depolarization, linear scalar retardance) obtained from Lu-Chipman decomposition⁶ of the recorded Mueller matrices of fixed human brain specimens, and exploring both statistical learning techniques and more modern, deep neural network approaches.

2. Instrumentation, Image Collection and Labeling

We used the custom-built multi-spectral wide-field imaging Mueller polarimetric system based on ferroelectric liquid crystals for ex-vivo polarimetric imaging of brain tissue. The details on the instrument design can be find elsewhere.^{7, 8} Sequential modulation of polarization of the probing light beam and the analysis of its polarization state upon reflection by a sample give access to the 16 images of Mueller matrix of a brain specimen at chosen wavelength. Non-linear compression of the recorded Mueller matrix images was applied pixel-wise to generate the maps of depolarization, linear scalar retardance and azimuth of the optical axis of linear birefringent medium.

We used the above-described system to image formalin-fixed human donor brains. A waiver for ethical approval for this purpose was obtained from the Ethics Committee of the Canton of Bern (KEK 2017-1189). Axial sections, measuring several centimeters in thickness, were measured at 550nm and 650nm. From each section several images of 3cm by 2.5cm were taken on both sides. Images from four brains were selected to be included in this study. These images were acquired from various locations in the brain, reflecting the variety of neuroanatomical features. Grey matter and background pixels (those containing neither grey matter nor white matter) were then manually segmented using Fiji.⁹ Images from one brain (”B3”) were set aside for evaluation of the machine learning algorithms: this yielded a set of 134 images in total for the evaluation of our algorithms. The remainder (”B1”, ”B2” and ”B4”) yielded 202 images in total, which were used for training.

3. Machine Learning Methodology

As a baseline for the ability of polarimetric parameters to separate grey and white matter in fixed human brain tissue, we built a pixelwise logistic regression model trained on all pixels arising from the grey or white matter (as indicated by the manual segmentation) of images from our three training brains. Input variables to the logistic regression were scalar retardance, depolarization, and wavelength (550nm or 650nm). To test the benefit of more sophisticated modelling, we trained a standard deep-learning-based segmentation model (Unet¹⁰) for the three class task of segmenting wide-field polarimetry images of the human brain into grey matter, white matter and background. Given the small amount of data available, we opted to use a “dynamic Unet”, as implemented in the fastai library,¹¹ in which a pretrained Resnet34 network¹² is leveraged to provide features computed at multiple scales for the purpose of segmenting the polarimetry image. To show the benefit of polarimetric data for this task, we trained two models on different input data: a model operating exclusively on normalized intensity images, and a model operating on a combination of intensity, depolarization and linear scalar retardance. Models were trained for 20 epochs, using the fit-one-cycle learning rate schedule with a maximum learning rate of 1e-4 and the Adam optimizer.

4. Results

An example of polarimetric data acquired from fixed human brain sections can be seen in Figure 1. While a distinction can be made seen between grey and white matter in both the depolarization and scalar retardance images, thresholding on either of these parameters is not in general a satisfactory method for identifying white matter: see Figure 2.

Figure 1. Images of a formalin fixed human brain taken from brain ”B3” using Lu-Chipman decomposition of data acquired from the Mueller widefield system.

(a) intensity image, (b) depolarization, (c) linear scalar retardance, (d) quiver plot of retardance, showing magnitude and azimuth, manually segmented (e) background pixels and (f) grey matter pixels.

Figure 2.

Receiver operating characteristic for thresholding on scalar retardance, thresholding on depolarization, the logistic regression model over scalar retardance, depolarization and wavelength, the Unet over scalar retardance, depolarization (‘all inputs’) and the Unet operating on the intensity image only.

A linear combination of these parameters is also outperformed by a simple Unet operating on intensity information only, but the best performance in distinguishing grey from white matter comes from a Unet combining intensity and polarimetric parameters (see Table 1).

Table 1. Performance of machine learning algorithms: detection of white matter.

Algorithm	Accuracy	Precision	Recall	ROC-AUC
Logistic Regression	0.74	0.82	0.69	0.81
Unet (intensity)	0.77	0.86	0.70	0.86
Unet (intensity, scalar retardance, depolarization)	0.84	0.91	0.78	0.92

5. Discussion

We have shown a marked superiority of deep learning models operating on polarimetric data for the task of white matter segmentation in fixed brain slices, versus pixel-wise thresholding, logistic regression, and deep models operating on intensity alone. By setting a rather high threshold for accepting pixels as white matter, the directions of white matter fibres can be readily observed without distraction from grey matter and background signals (see Figure 3 and compare to Figure 1 (d)). Since our ultimate goal is detection of fibre tracts in in-vivo tissue, our next goal is validation on unfixed tissue, in particular on surgically excised near-in-vivo tissue from neurosurgery. Here we will also examine the separation of tumor tissue from healthy-appearing white matter. Our hypothesis is that the uniformity of the retardance will play a crucial role here. Incorporating this may involve learning from the circular variance of the azimuth, or alternatively directly learning from the retardance as a vector field, for example by leveraging circular harmonics.¹³

Figure 3.

A quiver plot of retardance (same sample as shown in Figure 1, showing retardance only at pixels determined by the Unet as being of high probability (≥ 0.9) of belonging to white matter.