Identification of voxel-based texture abnormalities as new biomarkers for schizophrenia and major depressive patients using layer-wise relevance propagation on deep learning decisions

Abstract

Non-segmented MRI brain images are used for the identification of new Magnetic Resonance Imaging (MRI) biomarkers able to differentiate between schizophrenic patients (SCZ), major depressive patients (MD) and healthy controls (HC). Brain texture measures such as entropy and contrast, capturing the neighboring variation of MRI voxel intensities, were computed and fed into deep learning technique for group classification. Layer-wise relevance was applied for the localization of the classification results. Texture feature map of non-segmented brain MRI scans were extracted from 141 SCZ, 103 MD and 238 HC. The gray level co-occurrence matrix (GLCM) was calculated on a voxel-by-voxel basis in a cube of voxels. Deep learning tested if texture feature map could predict diagnostic group membership of three classes under a binary classification (SCZ vs. HC, MD vs. HC, SCZ vs. MD). The method was applied in a repeated nested cross-validation scheme and cross-validated feature selection. The regions with the highest relevance (positive/negative) are presented. The method was applied on non-segmented images reducing the computation complexity and the error associated with segmentation process.

1. Introduction

Schizophrenia (SCZ) and Major Depression (MD) are two of the most frequent psychiatric disorders (SCZ: 1.1% (Desai et al., 2013; Kessler et al., 2005; WU et al., 2006), MD: 7.1% of all U.S. adults [https://www.nimh.nih.gov/health/statistics/major-depression.shtml]), which furthermore often co-occur: around 40% of SCZ patients also suffer from MD (Upthegrove et al., 2016). Structural alterations in schizophrenia (SCZ) and major depressive (MD) patients mainly have been regarded as the result of a dysfunctional neurodevelopmental process with a system-specific involvement of higher-order cortical networks (Weinberger, 1987). In SCZ, volumetric reductions in gray-matter volume are reliably reported both for whole brain, and across large networks, differentially involving frontal, temporal, limbic, thalamic and striatal regions. Increased volumes in cerebrospinal fluid (CSF) structures are mentioned while greater white-matter disruptions were associated with greater symptom severity (Fornito et al., 2009; Haijma et al., 2013; Honea et al., 2005; Lener et al., 2015). MD in return, reduced gray volumes were reported in regions associated with emotion regulation such as the amygdala and the prefrontal cortex as well as sensorimotor areas (H. Zhang et al., 2016), i.e. the basal ganglia, thalamus, hippocampus, frontal lobe, orbitofrontal cortex, and gyrus rectus (Qiu and Li, 2018), CSF volumes were shown enlargement for global brain CSF and for lateral ventricles and left and right sylvian fissure regions (Cardoner et al., 2003). On the other hand, structural imaging studies have reported a significant increase of white matter hyper-intensities, which suggests that a change in water content in the frontal and parietal lobes occurs in patients with MD (Chen et al., 2016).

Texture analysis (TA) enables the quantification of the gray levels and brain patterns on MRI, via voxel inter-relations and spectral properties of the images (Kassner and Thornhill, 2010; Meyer et al., 2017). Each grey level represents a range of sample intensity values. In this study, all intensity values identified in every image were used. TA is a methodological approach widely used to MR image analysis for identification of tumor, (Zacharaki et al., 2009) and for differentiation between pathological and healthy tissue in different organs. Additionally, TA has established utility when applied to neuropsychiatric disorders including epilepsy (de Oliveira et al., 2013), multiple sclerosis (Theocharakis et al., 2009) and Alzheimer disease (J. Zhang et al., 2012). Recent studies indicate the potential for gray-matter texture measures to serve as candidate endophenotypes for Asperger syndrome (E Radulescu et al., 2013) and schizophrenia (Kovalev et al., 2003). For example, Radulescu et al. (Eugenia Radulescu et al., 2014) combined texture features with white-matter distribution reaching 70.6% accuracy in the classification of SCZ versus HC and measures of lower gray level homogeneity and higher entropy in patients compared to controls. On the other hand, in identification of artificially generated effects on real MRI data in Alzheimer’s Disease (AD) the variance has the best performance for the identification of these lesions, as it achieves a 100% correct detection rate in all types of artificial lesions (Maani et al., 2015). Based on author’s knowledge, it is the first time that TA is applied on MD subjects brain images.

In the current study, we examined potential differences between patients with SCZ or MD and HC by employing texture feature maps extracted from non-segmented MR images. The texture features have been widely used on tumor identification and recently a rapid increase has been in application on brain MRI for mental disorders. The extraction of texture feature maps from non-segmented brain images gives insights on the inter-relation of voxels from different modalities i.e., gray-matter volume, white-matter volume and cerebrospinal fluid. To our knowledge, non-segmented images have never been used for the recognition of these disorders, missing the interactions between these modalities as an indicator for the diagnosis that led to the identification of new biomarkers. In addition, this study should be considered as a multi-task analysis that additionally to the identification of biomarkers, deep white matter hyperintensities can be captured that are associated with clinical severity and treatment responsiveness. Common features of TA are second order statistical features, such as entropy, energy, homogeneity, contrast, correlation and variance that express the heterogeneity of the brain by measuring the inter-relations between voxels. In addition, difference of entropy, difference of variance, sum of variance and sum of average can be determined, reflecting second order statistics of differentiation of gray level distribution. This study aims to quantify the brain structural heterogeneity in subjects with schizophrenia and major depression by mapping regional brain alterations at the level of individual participants.

The deep learning technique utilize the registered MRI-feature based image as input. The output of the layer-wise relevant propagation (LRP) visualization is a 3D heatmap for each subject (Bach et al., 2015), indicating voxels that contribute significantly to the classification of the groups, in the established and advanced MRI biomarkers in SCZ and MD. A clustering algorithm is implemented for the visualization of the voxel relevancies in the testing set.

2. Methods

2.1. Study participants

In this study, sMRI scans of 482 subjects were used, 238 healthy controls, 141 schizophrenic (SCZ) patients and 103 major depressive patients (MD). The data came from the Department of Psychiatry and Psychotherapy, Ludwig-Maximilian University, Munich. Patient evaluations included: the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV): Axis I and II Disorders, the review of records and psychotropic medications as well as symptom assessments using the Positive and Negative Symptom Scale (PANSS) in SCZ group, the Scale for the Assessment of Negative Symptoms in SCZ group, and the Hamilton Depression Rating Scale in MD group. A consensus diagnosis was achieved by two experienced psychiatrists at study inclusion and after 1 year using the DSM-IV (for details see (N. Koutsouleris et al., 2014)). The initial data set was 288 healthy subjects, 141 schizophrenic subjects, 104 major depressive subjects and 57 bipolar and schizophrenic spectrum disorders that comprised the neuroanatomic data published in (N. Koutsouleris et al., 2014). Corrupted images were excluded from the analysis. The dataset extracted includes 240 males and 242 females, aged 18–65, and came from three diagnostic groups (see Table 1 for a sample description).

Table 1.

Sample description

	HC	SCZ	MD	Statistics (SCZ vs. HC)	Statistics (MD vs. HC)	Statistics (SCZ vs. MD)
N	238	141	103
Sex (male/female)	113/125	82/59	45/58	X2 = 14.783** p=.000	X2 = 1.031 p=.310	X2 = 33.799** p=.000
Age [yrs, M±1SD]	33.8±11.4	30.8±10.4	42.3±11.9	T=2.7**, p=.000	T=6.8**, p=.000	T=8.4**, p=.000

Note.

^**:p<.01, HC: healthy control, SCZ: schizophrenia, MD: major depression

2.2. MRI data acquisition and data preprocessing

MRI data acquisition was conducted in the Department of Psychiatry and Psychotherapy, Ludwig-Maximilian University, Munich, using a 1.5 Tesla Siemens MAGNETOM Vision Scanner. Imaging was performed with a 3-dimensional magnetization T1-weighted, prepared rapid-acquisition gradient echo sequence, including a relaxation time of 11.6 ms, time-to-echo of 4.9 ms, 126 contiguous axial sections of 1.5 mm thickness, matrix 512 × 512, voxel size 0.45 × 0.45 × 1.5 mm, field of view 230mm.

After inspection for artifacts and gross abnormalities, the images were segmented into gray-matter, white-matter, and cerebrospinal fluid tissue maps in native space by means of the Cat12toolbox (http://dbm.neuro.uni-jena.de), an extension of the SPM12 software package (Wellcome Department of Cognitive Neurology, London, England). The process was automated and has been previously applied in (N. Koutsouleris et al., 2014; Nikolaos Koutsouleris et al., 2009). The computation time of the preprocess was less than 30 minutes per subject. In detail, the toolbox extends the unified segmentation model (Ashburner and Friston, 2005) consisting of MRI field intensity inhomogeneity correction, spatial normalization and tissue segmentation at several preprocessing steps in order to further improve the quality of data preprocessing. Initially, the Optimized Blockwise Nonlocal-Means filter proposed by (Coupé et al., 2006) was applied to the MRI scans using the Rician noise adaption introduced in (Wiest-Daesslé et al., 2008) to increase the signal-to-noise ratio in the data. Then, an adaptive maximum a posteriori segmentation approach (Rajapakse et al., 1997) extended by partial volume estimation (Manjón et al., 2008) was employed to separate the MRI scans into gray-matter, white-matter, and cerebrospinal fluid tissue. The segmentation step was finished by applying a spatial constraint to the segmented tissue probability maps based on a hidden Markov Random Field model (Cuadra et al., 2005) that removed isolated voxels which were unlikely to be a member of a certain tissue class and closed holes in clusters of connected voxels of a certain class, resulting in a higher signal-to-noise ratio of the final tissue probability maps.

The p0* image produced via SPM12 segmentation, was used to mask the initial brain MRI to obtain a skull stripped T1 image in native space. Texture feature map were extracted from the masked image in the native space. The deformation field iy.* image produced via SPM12 was applied on the texture feature map matching the geometry of subject’s brain to standard space. Then, a dual registration strategy captured both large-scale and focal neuroanatomical variation in the study population: first, the texture feature maps were affinely registered to the single-subject Montreal Neurological Institute template, thus removing global brain volume differences across subjects while retaining large-scale, interindividual morphometric variation. Second, these maps were high dimensionally registered to the template using the DRAMMS algorithm which encoded focal, nonlinear tissue deformations in each individual brain relative to the template (see Fig. 1 for workflow). Affinely registered texture feature maps were resampled to 3mm isotropic voxels and entered the subsequent analysis (Ou et al., 2011). All scans were reviewed by a neuroradiologist to rule out clinically significant abnormalities.

Fig. 1.

Workflow for the calculation of three-dimensional gray level co-occurrence matrix.

2.3. Texture feature map

In the present study, we extracted textural parameters from non-segmented brain images by using gray-level co-occurrence matrix (GLCM). A co-occurrence matrix can be thought as a joint histogram of two random variables. In case of texture analysis applied on 3D MR images, the random variables are the gray level of one voxel g₁ and the gray level of its neighboring voxel g₂. The GLCM matrix was normalized by dividing the values with the total sum of the values in the matrix. The normalization was done for each GLCM matrix extracted in each 3D cube independently. The gray-level discretization method reduces the complexity on the calculation of the GLCM while had a direct impact on texture feature reproducibility. In this study, we did not discretize the intensity values, the total number of the intensities included in each 3D cube, instead of the whole brain, was used concluding in dimensionality reduction. A neighboring voxel is defined by a radius and a direction from another voxel. The GLCM was proposed by Haralick et al. (Haralick et al., 1973), is a texture feature map extraction method based on gray level spatial dependence. Principally, the GLCM is a measure of how often different combinations of pixel brightness values occur in an image. The existence of paired gray level values was calculated inside the cube. In this study, 7 texture feature maps were extracted from the gray level co-occurrence matrices on every brain MR image. The normalized 2D gray level co-occurrence matrix (GLCM) (Haralick et al., 1973), was calculated on a voxel-by-voxel basis in 3D images. A representation of how the GLCM is calculated is given in Fig. 1.

In detail, a voxel-by-voxel sliding 3D cube of 7 × 7 × 7 dimension was centered in every voxel. The number of the gray levels [n] in each cube is the dimension of the 2D GLCM [nxn]. So, for every cube, a 2D GLCM was constructed and the specific feature maps were extracted. The GLCM was normalized dividing with the sum of the values of the GLCM. The selected feature maps were calculated in every GLCM and stored as a 3D image.

All the feature maps calculated were basically a function of the probability of each GLCM entry and the difference of the gray levels, g₁ and g₂ (Eichkitz et al., 2013). Only for cubes includes non-zero values, the feature maps were calculated as presented in Appendix A.

The registered texture feature maps on the MNI space were fed into deep learning technique for group classification. An example of 2D images for each feature map for one subject is presented in Fig. 2.

Fig. 2.

The registered texture feature map for the significant features across a subject: a) contrast, b) entropy, c) variance, d) difference of entropy, e) difference of variance, f) sum of variance and g) sum of average.

2.4. Deep learning

In this study, 10 times repeated nested cross-validation scheme with 10 folds in the inner cycle and 10 folds in the outer cycle (namely 10 × 10) was implemented with 3 hidden layers, each hidden layer consists of 20 nodes and 1,000 epochs, see Fig. 3. Network parameters selected after experimentation based on the trial-and-error process. Three hidden layers were selected as the minimum deep learning network established with low complexity. The number of nodes was selected to 20, regarding the complexity and after experimented from 5 to 50 nodes with step 5. The dataset splits into the 10 folds. 9 out of 10 folds are used for training the network and the one remaining fold contains the hold-out dataset. The training set splits into 10 folds, 9 out of 10 are used for the training of the network and the rest for the validation. During the validation, the selection of parameters is taking place and the winner model is selected. As the parameter selection and, finally, the winner model is selected, the hold-out dataset is fed into the network. This process is repeated 10 times by shuffling the initial dataset. The average accuracy of the hold-out dataset across the 10 × 10 nested cross-validation, repeated 10 times, is finally calculated.

Fig. 3.

Nested cross-validation scheme.

The classifier developed in this project was a neural network-based classifier implemented in MATLAB (MathWorks Inc., Natick, Massachusetts, USA). The three-layer back propagation neural network comprises of an input layer, an output layer and two hidden layers. The network used the hyperbolic tangent sigmoid transfer function and was batch trained using the Levenberg-Marquardt training algorithm (J.-L. Zhang, 2003). The main way of accessing possible types of uncertainty in deep learning is using dropout as regularizer to avoid overfitting. Dropout consists of randomly sample network nodes and drop out during training randomly, a fraction p of nodes and corresponding activations. In testing phase, all activations used are reduced by the same factor p. The early stopping hyperparameter selection method was applied to automatically determine the correct amount of regularization. L2-regularization is applied as constraint to avoid overfitting/underfitting while dropout disable random nodes during training time, so a different subset of the network architecture is evaluated and adjusted with each training step. The use of deep neural networks has the advantage of giving the probability for each subject to belong to a class. Cross-entropy loss measures the performance of a classification model whose output is a probability value between 0 and 1. Cross-entropy loss increases as the predicted probability diverges from the actual label. Sensitivity and specificity are calculated to ensure good classification performance for both groups. The dataset is randomly separated in a 10 × 10 nested classification scheme with 10 times repetition concluding in 10,000 models. Feature selection method (two-sample t-test), in the inner cycle, was cross-validated selected a comparable number of features with the dimension of the database, namely, the top 200 ranked features that best discriminate the 2 classes (Zanetti et al., 2013). The way that these interconnections developed is presented in the study (Bach et al., 2015) and the method is implemented here.

2.5. Visualization and evaluation of heatmaps

In order to perform localization, we calculated the relevance of the voxels in each class using the LRP algorithm for multilayer neural network, as described in (Bach et al., 2015). The explanation given by LRP would be a map of which voxels in the original texture feature map contribute to the diagnosis and to what extent.

For the specific deep learning scheme with 3 hidden layers and size 20, the calculation of LRP algorithm is presented in Appendix B. The output of the LRP algorithm is a heatmap for each subject uncovering change on brain structures for both SCZ and MD patients. The final images were smoothed with a Gaussian filter of 10mm, and these are visualized by using the MRIcron toolbox (https://people.cas.sc.edu/rorden/mricron/install.html). Similarities of the heatmaps for subjects in the same group were identified by clustering the heatmaps. Evaluation of the clustered heatmaps gives an estimation to the importance of each brain region to the classification decision. Images in Figure 4 were produced by averaging the heatmaps of the subjects in each cluster for each group and overlay in an initial image in MRIcron toolbox.

Fig. 4.

We demonstrated the relevancies of the correct classified subjects of each group against the other in each classification scheme for the registered texture feature map: a) contrast, b) entropy, c) variance, d) difference of entropy, e) difference of variance, f) sum of variance and g) sum of average. The red (cluster 1), blue (cluster 2) and green (cluster 3) color corresponds to the sorted clusters according to the number of subjects.

2.6. Clustering of subjects

The aim was to display the heatmap of each correctly classified subject in the hold-out testing set produced by the LRP algorithm in a grouped fashion (Stelzer et al., 2019). The affinity propagation (AP) algorithm (Frey and Dueck, 2007) was selected to cluster the subject’s positive relevance, which uses the concept of message passing between the samples. The main advantage of the AP algorithm is that the number of clusters is not predefined. The input in the clustering algorithm is a matrix M × N, where N is the number of subjects and M is the relevance of each voxel. The output of the AP algorithm is a scalar for every subject express in which cluster the subject belongs to. The average of the heatmaps from the subjects belong to each cluster are represented.

For each classification process the relevancies were classified using the AP algorithm. The clusters are presented in Fig. 4, for each texture feature map under the three-classification scheme. The red color corresponds to the most intense cluster, the second one is in blue color and the green is the third one, if exist (see Fig. 4).

3. Results

3.1. Classification results and visualization

Three classifiers were developed in this study: a) SCZ vs. HC, b) MD vs. HC and c) SCZ vs. MD. The average accuracy, sensitivity and specificity in the outer cycle were calculated for 10 folds in 10 repetitions. We tested the texture feature maps one-by-one in the deep learning classifier (see Table 2, 3 and 4). Sensitivity and specificity values are closed enough for the a) and c) classification schemes. On the other hand, the identification of MD against HC is not so accurate, while the specificity and sensitivity values of this classification process differ for more than 4%. The most discriminative feature for the classification of SCZ vs HC is the contrast map resulting in both high sensitivity and specificity. For MD recognition against HC the specificity remains low that depicts the correct classification of HC, but two texture feature maps, the difference of entropy and the variance give promising results. For the discrimination of MD vs SCZ the difference of entropy map recognizes both groups with almost 80% accuracy. Due to computational costs the combination of texture feature maps is not performed in this analysis. It has been checked in a smaller database that the combination of the texture feature maps does not improve the classification accuracy. However, the aim was to investigate the interpretability of each texture feature map separately.

Table 2.

presents the accuracy, sensitivity and specificity of the hold-out testing set, for the classification between SCZ and HC.

Registered texture feature map	Accuracy	Sensitivity	Specificity
Variance	84.6%±5.8%	82.7%±5.1 %	80.6%±10.4%
Entropy	84.1%±5.4%	83.0%±5.0%	82.6%±10.1%
Contrast	84.5%±5.4%	83.9%±5.6%	84.5%±10.6%
Difference of entropy	84.1%±5.5%	82.7%±5.3%	81.2%±10.7%
Difference of variance	83.5%±5.7%	82.1%±5.2%	81.2%±10.9%
Sum of variance	83.4%±5.6%	81.9%±5.2%	80.2%±10.4%
Sum of average	84.4%±5.0%	82.5%±4.7%	80.7%±10.8%

Table 3.

presents the accuracy, sensitivity and specificity of the hold-out testing set, for the classification between MD and HC.

Registered texture feature map	Accuracy	Sensitivity	Specificity
Variance	86.2%±5.0%	83.9%±5.1%	78.7%±13.1%
Entropy	86.1%±5.1%	83.8%±5.3%	78.4%±14.4%
Contrast	86.2%±4.7%	83.5%±5.8%	76.3%±14.6%
Difference of entropy	86.7%±5.6%	84.3%±5.5%	78.5%±14.4%
Difference of variance	86.2%±5.3%	83.6%±5.3%	76.3%±13.4%
Sum of variance	85.9%±5.5%	83.0%±5.7%	76.2%±14.9%
Sum of average	85.6%±4.7%	83.5%±5.1%	77.8%±12.7%

Table 4.

presents accuracy, sensitivity and specificity the hold-out testing set, for the classification between MD and SCZ.

Registered texture feature map	Accuracy	Sensitivity	Specificity
Variance	76.19%±12.15%	76.92%±10.62%	72.73%±17.98%
Entropy	78.86%±12.12%	79.08%±7.31%	81.21%±8.09%
Contrast	75.84%±9.79%	77.99%±6.31%	82.2%±8.55%
Difference of entropy	80.94%±9.43%	80.18%±5.86%	80.66%±6.33%
Difference of variance	75.80%±9.92%	78.28%±5.25%	81.44%±5.91%
Sum of variance	74.5%± 10.80%	78.27%±8.41%	82.75%±10.14%
Sum of average	73.81%±10.14%	76.78%±8.39%	80.99%±8.39%

3.2. Visualization of clustering of subjects

Across texture feature maps, patients with SCZ has higher relevance compared to HC and MD, in thalamus (i.e. SCZ>MD: variance, contrast, difference of variance and sum of average), cerebral white matter (i.e. SCZ>HC: variance, difference of variance, SCZ>MD: variance, contrast, difference of entropy, difference of variance, sum of variance, sum of average), hypothalamus (SCZ>HC: variance, sum of variance, difference of variance and sum of average), nucleus accumbers (SCZ>MD: variance, difference of entropy), ventral and posterior cingulate cortex and gyrus rectus (SCZ>HC: variance, entropy, difference of variance, sum of variance, contrast, SCZ>MD: contrast). Regarding the LRP explanation, variance is a dominate texture feature map for the identification of schizophrenia against both HC and MD, while difference of variance is the dominate texture feature map for the identification of schizophrenia against healthy controls. Contrast is the most promising texture feature map for the discrimination of schizophrenia from major depression following by difference of variance and sum of average.

In return, MD shows higher relevance in brain stem (MD>HC: variance, entropy, difference of variance, sum of variance and sum of average), cerebral white matter (MD>HC: entropy, MD>SCZ: variance, difference of entropy, sum of average) and third ventricle (MD>HC: all texture feature maps). Regarding the LRP explanations, difference of variance and sum of average are the dominate texture feature maps for major depression identification, while cerebral white matter abnormalities in major depression compared to healthy controls are captured from the entropy. Major depression is differentiated from schizophrenia in cerebral white matter according to variance, difference of entropy and sum of average.

4. Discussion

In this study, we used texture feature maps for the classification of participants with SCZ, MD patients and HC. The main scope of the study is to point out how the radiomics can help on identification of mental diseases by investigating how the geometric features asses brain-shape variation. Moreover, it is pointed out the dynamic of each texture feature map, as even using only one of the Haralick features (i.e., variance) can correctly discriminate the groups, reducing the computational costs. However, the implementation of the algorithm needs parameter selection. The selection of the radius was based on the study (Maani et al., 2015). The neighborhood radius and the directions should be large enough to be able to distinguish texture patterns, while small enough to detect local changes around each voxel. In our case, a radius of 3 voxels was applied in 3 directions. Also, the GLCM was calculated for the number of gray levels in each cube instead of calculating it on the whole gray-level number, reducing the computation time. In the context of texture features, a main issue is that MRI intensities are non-standardized and a large variability in image intensities could highly affect the extraction of the texture features. In this study, the texture feature maps extracted from the masked T1-weighted image in native space after brain extraction and the feature map intensities were registered in the MNI space. As the intensities of brain MRI in the native space cannot be comparable across patients, this process may be considered as a limitation of the study. However, there are multiple pre-processing pipelines for the standardization of MRI intensities, there is no consensus within texture features extraction process regarding the applied image normalization method (Carré et al., 2020). Additional geometrical features should be examined, as well as application of the algorithm for the classification of different groups of patients is in the future plans.

Regarding the clinical plausibility of results, first studies applying texture analysis on MR data in patients with SCZ revealed that heterogeneity in brain stem and ventricular regions was increased in first episode patients with schizophrenia (Latha and Kavitha, 2018) and uniformity as well as entropy were altered in the hippocampus, the amygdala and the cerebellum in chronic SCZ patients (Eugenia Radulescu et al., 2014). Despite of hippocampus and amygdala, current analyses supported these findings. In addition, structural alterations have been reported for multiple brain regions including the identified regions in the current analysis (temporal regions: (Liang et al., 2019), frontal cortex: (Pantelis, 2005; Torres et al., 2016; Zhao et al., 2018); striatum: (Egloff et al., 2018; Fan et al., 2019), hypothalamus (Tognin et al., 2012). Interestingly, positive relevance in ventral striatum for SCZ subjects are depicted in all texture-based images in line with previous studies (Juckel et al., 2006; Kirschner et al., 2016). Furthermore, the localization of alterations seemed to vary between stages of illness: whereas temporal and PFC alterations were most prominent in the stage from high-risk state to conversion, first episode schizophrenic patients demonstrated predominantly declined frontal and subcortical volumes. In patients with chronic SCZ, in return, alterations were found to extended with disease duration and in function of clinical outcome (for review see (Dietsche et al., 2017). In the current analysis, chronic patients were analyzed, thus, extensive alterations are in line with these earlier findings.

For MD, Spuhler et al. (Spuhler et al., 2018) determined entropy from DTI images in a classification study with patients with bipolar depression and found that only entropy in the temporal pole was reduced in patients (classification accuracy: 83%). Using the texture feature maps, we were able to replicate this finding as well as for difference of entropy. However, in line with morphometrical studies, alterations in the OFC (Carlson et al., 2015; Palaniyappan et al., 2015; Webb et al., 2014) and the PFC (Goveas et al., 2011; Taki et al., 2005), were detected by texture feature maps. Recent advances in neuroimaging have reported widespread structural abnormalities, suggesting a dysfunctional frontal-limbic circuit involved in the pathophysiological mechanisms of depression (Coloigner et al., 2019). The third ventricle and aqueduct of sylvious are shown positive relevance for MD in comparison with HC in all texture-based images in consistence with previous studies (Kontoangelos et al., 2013; Palaniyappan et al., 2015). In addition, positive relevance of MD against SCZ are located in cerebellum in all images except from entropy and difference of entropy while positive relevance is presented in cerebellum for SCZ patients against MD for variance, difference of variance and sum of average in line with previous studies for the role of the cerebellum in schizophrenia and depression (Depping et al., 2018; Kirschner et al., 2016). In (Wagner et al., 2017), abnormalities in brain stem for major depression patients are presented and validate the present study’s results.

In (Surbeck et al., 2020), white matter tract integrity in patients with schizophrenia disorders and healthy controls was compared using diffusion tensor imaging combined with probabilistic fiber tractography. Multivariate analyses of fractional anisotropy, mean, axial, and radial diffusivity of the left inferior fronto-occipital fasciculus (IFOF) showed significant differences between SCZ patients and controls. In agreement with this study, entropy and difference of entropy images reveal positive relevance on the IFOF for MD vs. SCZ patients, and the difference of entropy image for SCZ vs. MD. Anterior corona radiata changes captured by variance and difference of variance, entropy, sum of variance and sum of average in SCZ patients against HC that is in line with previous studies indicated differentiations between the fractional anisotropy and the radial diffusivity in the anterior corona radiata in patients with schizophrenia against healthy controls (Koshiyama et al., 2018).

Studies of brain white matter volume changes, connectivity disruptions, as well as genetic factors affecting myelination can throw light on the nature of white matter abnormalities underpinning MD (Harris et al., 2020; Shen et al., 2020; Tham et al., 2011). Several studies suggest the patients with late-onset major depression present an increased load of cerebral white-matter lesions compared with age-matched controls (Dalby et al., 2010). Consistent with previous studies, reduced white matter was observed in the genu of the corpus callosum extending to the inferior fasciculus and posterior thalamic radiation, confirming a frontal-limbic circuit abnormality in (Coloigner et al., 2019). Structural magnetic resonance imaging studies have revealed deep white matter hyperintensities which are associated with clinical severity, and treatment responsiveness in MD patients. In total, a high relevance in cerebral white matter between SCZ and both HC and MD are presented in variance, difference of variance, entropy and difference of entropy images.

Changes in the volume of brain white matter has been reported in schizophrenia and bipolar disorder, with reductions in white matter of schizophrenic patients are largely driven (94%) by genetic factors (van Haren et al., 2012). The polygenic schizophrenia-related risk scores can be used as biomarkers to predict outcomes in schizophrenia, as indicated by many studies, there is an association between white matter volumes and polygenic risk scores (Alloza et al., 2018; Harrisberger et al., 2018; Jonas et al., 2019; Simões et al., 2020). In (Whalley et al., 2013), authors report that a higher polygenic risk allele load for MD was significantly associated with decreased white matter integrity in a large cluster, with a peak in the right-sided superior longitudinal fasciculus. Further investigation of the impact of genetic factors on white matter abnormalities is in author’s future plans.

The variability of the results among groups is enhanced by the differentiation of the texture feature maps selected for this analysis, but also from the huge variability of the antipsychotic treatment among patients and illnesses. The effect of antipsychotics and antidepressant treatments in volumetric abnormalities is remain a controversial debate. Studies have shown that antipsychotic and antidepressant treatments have an impact on alterations in white matter disturbances between treated patients, drug-naïve and healthy controls (Kubicki and Lyall, 2018; Szeszko et al., 2014; van Velzen et al., 2020). Smaller brain tissue volumes and larger cerebrospinal fluid volumes is observed in long treatment with antipsychotics (Ho et al., 2011). In (Dusi et al., 2015) increased right hippocampal volumes have been found in female responders compared to non-responders after eight weeks of fluoxetine treatment, while in this study hippocampus is not considered a potential biomarker.

However, one key region of both, SCZ and MD, did not appear in the current findings, i.e., the medio-temporal lobe including the hippocampus and the amygdala. Ventral striatum is a key region for schizophrenia and cerebral white matter is a key region for major depression. The third ventricle and aqueduct of sylvious are key regions for major depression recognition from healthy subjects. Instead, dominated texture feature maps were revealed for the identification of schizophrenia and major depression. In this study indicated that variance is the key feature for the recognition of schizophrenia. The discrimination of schizophrenia from healthy controls (major depression) is proven by the difference of variance feature (contrast). On the other hand, major depression identification lays on difference of variance and sum of average features, while the entropy feature reveals the major depression on cerebral white matter in contrast to healthy controls. Cerebral white matter is key region for the differentiation of major depression from schizophrenia by the variance, difference of entropy and sum of average features.

Based on the author’s knowledge, it is the first time that texture feature map on non-segmented 3D MR images for the classification of SCZ and MD were implemented. Comparison with other established methods such as convolutional neural networks are also in author’s future goals. However, the effect of the parameter selection on the classification accuracy, such as the radius and the direction of neighboring voxels in feature extraction method, has to be further examined. Future study also would be the investigation of how the different clusters of the positive relevance in each group are correlated with clinical information.

5. Conclusions

To summarize, these findings suggest that texture feature map can be a useful representation for characterizing dissimilarities in brain structure that is complementary to volumetric analysis. Unfortunately, to date, the number of studies using texture analysis in the context of psychiatric disorders is low, and potential methodological restraints are not well enough understood to formulate potential (in)sensitivities towards MR image quality (e.g., cortical vs. subcortical structure, WM vs. GM vs. CSF etc.). Therefore, it is of highest interest, to address brain alterations using parameters beyond brain morphometry and function such as texture-based features in future studies. Future studies are necessary to more systematically investigate the relationship between manifested neurobiological markers and LRP explanations.

Appendix

Appendix A

Entropy: measure the complexity of the texture distribution. Entropy is a measure of chaos, if the values are consistently across, this means that the texture is very stochastic. Inverse to this property is the energy.

$$Entropy=-\sum _{i=1}^{N_{g_1}}\sum _{j=1}^{N_{g_2}}GLCM(i,j)\ln GLCM(i,j)$$

Contrast: reflects the distance from the GLCM diagonal. Values on the diagonal (where i and j are the same) result in zero contrast, whereas the contrast increases by increase of distance from the diagonal.

$$Contrast=\sum _{i=1}^{N_{g_1}}\sum _{j=1}^{N_{g_2}}{(i-j)}^{2}GCCM(i,j)$$

Variance: is the variance of the intensities of all reference voxels in the relationships that contributed to the GLCM.

$$Variance=\sum _{i=1}^{N_{g_1}}\sum _{j=1}^{N_{g_2}}{(i-\mu )}^{2}GLCM(i,j)$$

Where μ is the GLCM mean.

Difference of Entropy: measures the disorder related to the gray level difference distribution of the image.

$$Dif{f}_{entropy}=\sum _{k=0}^{N_g-1}\sum _{i=1}^{N_{g_1}}\sum _{j=1}^{N_{g_2}}GLCM(i,j){\log }_2\left(GLC{M}_{x-y}(k)\right)$$

where GLCM_x−y, N_g are expressed as:

$$GLC{M}_{x-y}(k)=\sum _{i=1}^{N_{g_1}}\sum _{j=1}^{N_{g_2}}GLCM(i,j),where|i-j|=k,andk=0,1,\mathrm{..},N_g-1$$

Difference of Variance: is a measure of heterogeneity that places higher weights on differing intensity level pairs that deviate more from the mean.

$$Diff\_Variance\text{=}\sum _{k=0}^{N_g-1}{(k-Diff\_Average)}^{2}GLC{M}_{x-y}(k)$$

Sum of Average: measures the mean of the gray level sum distribution of the image.

$$Sum\_Average\text{=}\sum _{k\text{=2}}^{\text{2}{N}_g}kGLC{M}_{x+y}(k)$$

where GLCM_x+y, N_g are expressed as:

$$GLC{M}_{x+y}(k)=\sum _{i=1}^{N_{81}}\sum _{i=1}^{N_{X2}}GLCM(i,j),wherei+j=k,andk=2,3,\mathrm{..},N_g$$

Sum of Variance: measures the dispersion (with regard to the mean) of the gray level sum distribution of the image.

$$Sum\_Variance\text{=}\sum _{k=2}^{2N_z}{(k-\mu )}^{2}GLC{M}_{x+y}(k)$$

Appendix B

For the specific deep learning scheme with 3 hidden layers with size 20, the LRP algorithm is presented:

Relevance of the 5^th Layer

$${{\mathit{R}}_{\mathit{j}}}^{(5)}=f(x),j=1,2$$

Where the fourth layer is the real-valued prediction output of the classifier f for the two classes j.

Relevance of the 4^th Layer between neurons i and j

For j = 1,2 and i = 1,…,20

$${{\mathit{R}}_{\mathit{i}\leftarrow \mathit{j}}}^{(4,5)}=\{\begin{array}{l}\frac{z_{ij}}{z_j+\epsilon }{R}_j^{(5)},z_j\ge 0\hfill \\ \frac{z_{ij}}{z_j-\epsilon }{R}_j^{(5)},z_j<0\hfill \end{array},$$

z_ij = x_iw_ij,

$$z_j=\sum _{i=1}^{20}{z}_{ij}+b_j$$

Where x_i is the output of the third hidden layer using the tansig transfer function on the net input, w_ij are the weights and b_j the biases of the neurons connect the fourth and third layer. ε is 0.001 just to avoid the division with zero. So, the voxel-wise relevance in the third hidden layer is calculated as:

$${{\mathit{R}}_{\mathit{i}}}^{(4)}=\sum _{j=1}^2{R_{i\leftarrow j}}^{(4,5)}$$

Relevance of the 3^nd Layer between neurons i and k

For i = 1,..,20 and k = 1,…,20

$${{\mathit{R}}_{\mathit{k}\leftarrow \mathit{i}}}^{(3,4)}=\{\begin{array}{l}\frac{z_{ki}}{z_i+\epsilon }{R}_i^{(4)},z_i\ge 0\hfill \\ \frac{z_{ki}}{z_i-\epsilon }{R}_i^{(4)},z_i<0\hfill \end{array},$$

z_ki = x_kw_ki,

$$z_i=\sum _{k=1}^{20}{z}_{ki}+b_i$$

Where x_k is the output of the second hidden layer using the tansig transfer function on the net input, w_ki are the weights and b_i the biases of the neurons connect the second and third layer. ε is 0.001 just to avoid the division with zero. So, the voxel-wise relevance in the second hidden layer is calculated as:

$${{\mathit{R}}_{\mathit{k}}}^{(3)}=\sum _{i=1}^{20}{R_{k\leftarrow i}}^{(3,4)}$$

Relevance of the 2^nd Layer between neurons k and l

For k = 1,..,20 and l = 1,…,20

$${{\mathit{R}}_{\mathit{l}\leftarrow k}}^{(2,3)}=\{\begin{array}{l}\frac{z_{lk}}{z_k+\epsilon }{R}_k^{(3)},z_k\ge 0\hfill \\ \frac{z_{lk}}{z_k-\epsilon }{R}_k^{(3)},z_k<0\hfill \end{array},$$

z_lk = x_lw_lk,

$$z_k=\sum _{l=1}^{20}{z}_{lk}+b_k$$

Where x_l is the output of the first hidden layer using the tansig transfer function on the net input, w_lk are the weights and b_k the biases of the neurons connect the second and third layer. ε is 0.001 just to avoid the division with zero. So, the voxel-wise relevance in the first hidden layer is calculated as:

$${{\mathit{R}}_{\mathit{l}}}^{(2)}=\sum _{k=1}^{20}{R_{l\leftarrow k}}^{(2,3)}$$

Relevance of the 1^st Layer between input voxels and neurons l

For d = 1,…,212295 voxels:

$${{\mathit{R}}_{\mathit{d}\leftarrow \mathit{l}}}^{(1,2)}=\{\begin{array}{l}\frac{z_{dl}}{z_l+\epsilon }{R}_l^{(2)},z_l\ge 0\hfill \\ \frac{z_{dl}}{z_l-\epsilon }{R}_l^{(2)},z_l<0\hfill \end{array},$$

z_dl = x_dw_dl,

$$z_l=\sum _{d=1}^{212295}{z}_{dl}+b_l$$

Where x_d is the input registered texture feature map based image, w_dl are the weights and b_l the biases of the neurons connect the input and second layer. So, the voxel-wise relevance in the input layer is calculated as:

$${{\mathit{R}}_{\mathit{d}}}^{(1)}=\sum _{l=1}^{20}{R_{d\leftarrow l}}^{(1,2)}$$