Predicting Cognitive Scores from Resting fMRI Data and Geometric Features of the Brain

Abstract

Anatomical T1 weighted Magnetic Resonance Imaging (MRI) and functional magnetic resonance imaging collected during resting (rfMRI) are promising markers that offer insight into the structure and function of the human brain. The objective of this work is to explore the use of a deep learning neural network to predict cognitive performance scores for a population of normal controls and subjects with Attention Deficit Hyperactivity Disorder (ADHD). Specifically, we predict verbal and performance IQs and ADHD index from features derived from T1 and rfMRI imaging data. First, we processed the rfMRI and MRI data of subjects using the BrainSuite fMRI Processing (BFP) pipeline to perform anatomical and functional preprocessing. This produces for each subject fMRI and geometric (anatomical) features represented in a standardized grayordinate system. The geometric and functional cortical data corresponding to the two hemispheres were then transformed to 128×128 multichannel images and input to a convolutional component of the neural network. Subcortical data were presented in a standard vector form and inputted to a input layer of the network. The neural network was implemented in Python using the Keras library with a TensorFlow backend. Training was performed on 168 images with 90 images used for testing. We observed a high correlation between predicted and actual values of the indices tested: Performance IQ: 0.47; Verbal IQ: 0.41, ADHD: 0.57. Comparing these values to those from network trained on functional-only and structural-only data, we saw that rfMRI is more informative than MRI, but the two modalities are highly complementary in terms of predicting these indices.

1. INTRODUCTION

Anatomical (T1-weighted) and functional MRI (fMRI) techniques offer insight into the structure and function of the human brain. Multiple studies have been able to identify brain regions associated with a range of cognitive performance indices.¹ However, prediction of cognitive scores based on these MRI imaging techniques remains a challenging task which has attracted attention recently.^2,3 It is largely accepted that the brain is topographically organized into distinct areas that interact with each other as networks.⁴ Several studies have focused on the structural segregation or parcellation of the cerebral cortex into different areas, based on MRI data including T1 weighted MRI, fMRI, and diffusion-weighted MRI.¹ Functional MRI studies can then be used to associate these parcels with different aspects of cognitive performance. Here, rather than focus on identifying localized structural or functional facets of MRI data that are associated with specific aspects of cognitive performance, we apply a deep learning framework to use data from the entire brain to predict or infer various performance indices. The neural underpinnings of individual differences in intelligence are not well understood and are a matter of intense study.^5,6 The degree to which different features enable us to predict different performance indices may help us improve our understanding of the relationship between high-level cognitive function, and brain functional networks and structure as revealed respectively in resting and structural MRI studies.

The modelling of brain function as a Bayesian machine, in which brain areas are seen as connected and relatively specialized computational units, is being contradicted with the recent findings about functional specialization which show rich and heterogeneous patterns of behavioral function for many brain regions as well as distributed patterns involving multiple brain regions for many brain functions.^4,7 The view of functionally specialized brain regions and networks view is being revised in light of the emergence of the degeneracy principle that shows that one unique behavioral output or outcome can be achieved by different neurocognitive systems.⁸ The relationship between brain and cognition is therefore highly complex and difficult to model and identifying specific brain regions and networks or a set of regions and networks associated with cognitive performance is challenging.

The recent advances in machine learning, especially deep learning, indicate the ability of deep neural networks to reveal and learn such complex, distributed and heterogeneous relationships from multivariate data. The complex relationship between brain structure and function, as well as its association with the cognitive performance indices, could be learned using a data-driven machine learning approach.⁹

Here we proposed a deep learning approach that uses rfMRI data and geometric features extracted from anatomical T1 weighted MRI images to predict three indices: verbal IQ, performance IQ and Conner ADHD index¹⁰ in a group that includes controls and subjects diagnosed with ADHD. We train a neural network to predict these performance scores using three approaches: using only the structural features derived from T1 images, using only rfMRI data, and jointly using structural features derived from T1 images as well as rfMRI data.

2. MATERIALS AND METHODS

We assume an input consisting of T1-weighted MRI and rfMRI data for the study population. We also assume cognitive performance scores are available for the study subjects. Our goal is to use convolutional-neural-network based machine learning algorithms that takes preprocessed T1 and rfMRI data from subjects as inputs to predict cognitive scores as output. In order to use the neural network, we need to map data from individual subjects into a common coordinate system. We use the grayordinate representation¹¹ of the data for representing anatomical and functional features. This representation was generated using the BrainSuite fMRI pipeline (BFP).¹²

2.1. Data and preprocessing

The data for the study consisting of 259 subjects (typically-developing controls: 146, ADHD combined: 46, ADHD inattentive: 66, ADHD hyperactive: 1) was collected as a part of ADHD 200 competition and is available for download through the project website (http://fcon_1000.projects.nitrc.org/indi/adhd200). In this study, we did not subdivide the population according to subgroups but used their ADHD indices as a cognitive measure to be predicted. We used the data collected at the Peking University.^13,14 Images were acquired using a Siemens Trio 3-Tesla scanner. All of the resting-state functional data were acquired using echo-planar imaging (EPI) sequence. The scan parameters are described on the project website for different subsets.

We used the BFP to process the rfMRI subject data and generated grayordinate representations^15,16 of the preprocessed rfMRI signals. The BFP is a software workflow that processes fMRI and T1 data using a combination of software that includes BrainSuite (https://brainsuite.org), AFNI (https://afni.nimh.nih.gov), FSL (https://www.fmrib.ox.ac.uk/fsl), and MATLAB scripts to produce processed fMRI data represented in a common grayordinate system. Unique features of the BFP pipeline include cortically-constrained volumetric registration,^17,18 Global PDF-based non-local means filtering (GPDF)¹⁹ and BrainSync alignment of resting fMRI time series.^20,21 Starting from raw T1 and fMRI images, BFP produces processed fMRI data represented both on surface and volume co-registered with BrainSuite’s BCI-DNI atlas as well as a grayordinate based representation. For the grayordinate representation, the cerebral cortex is modeled as a surface mesh, whereas the globular subcortical nuclei are modeled as volume parcels. The grayordinates is a common space containing both cortical surface vertices and subcortical volume voxels.¹¹

2.1.1. Functional (rfMRI) data representation

We used the BrainSync transform,^20,21 a method for temporal synchronization of fMRI data across subjects, to align rfMRI data in the subject population to a representative subject. This was followed by a dimensionality reduction along the temporal dimension to 21, using PCA, with the basis chosen from the average signal from all subjects. For each subject, this results in a 21-dimensional vector representing functional (temporal) features at each grayordinate point.

2.1.2. Structural (MRI) data representation

We represented the geometric features of the brain-derived from T1 weighted MRI images in the same grayordinate representation. In order to generate geometric features, we computed the shape index (S), curvedness (C) and cortical thickness (T)²² as described below. As a part of the BFP processing sequence, the T1-weighted brain images were processed using BrainSuite to generate inner, mid and pial cortical representations. We computed cortical thickness using the method described in Joshi et al. 2018.²³ We modeled the geometry of the mid-cortical surface through the local surface-patch characteristics at each point on the surface mesh. We computed principal curvatures at each point on the mid-cortex by fitting a quadratic patch to the local surface around that point. The principal curvatures describe the local geometry of the cortex up to second order modulo rotation and translation.²⁴ The two principal curvatures are represented using orthogonal bases as Curvedness (C) and shape index (S) which meaningfully separate notions of bending and shape leading to an easier interpretation of the local geometry.²⁵ The features were generated on the mid-cortex at 5 different smoothness levels, resulting in a 15-dimensional geometric feature at each point in the cortex. These features were mapped to the grayordinate system as illustrated in Figure 1.

Figure 1.

(Left) fMRI data mapped to cortex (upper) and subcortical regions (lower) for grayordinates. (Right) Shape index (S), curvedness (C), and thickness (T).

As input to the neural network model, we mapped the cortical hemispheres from the grayordinate systems to unit squares of size 128 × 128 as shown in Figure 2. This mapping was performed using p-harmonic maps²⁶ with p = 4. This allows us to map grayordinate data (both functional data from rfMRI images as well as geometric features extracted from structural MRI images) as multi-channel images of size 128×128×C where C represents the number of channels. As explained earlier, we have 21 rfMRI channels and 15 structural channels. Additionally, the data from subcortical gray matter points are also input to the neural network as a multi-dimensional vector at each grayordinate point. This forms the input of size 128 × 128 ×C (two hemispheres) plus 31870 ×C (subcortical) data as input, where C = 15 for geometric features-based prediction, C = 21 for fMRI-based prediction and C = 21 + 15 = 36 for the combined prediction model. We used a neural network inspired by the VGG model²⁷ that uses a combination of convolutional, fully-connected, dropout, pooling and flattening layers with ReLu activations. In addition to the convolutional layers from the VGG model, we also added a fully connected neural network model for the subcortical representations as shown in Figure 3. The neural network was implemented in Python using the functional API in Keras²⁸ with TensorFlow²⁹ back-end. The Adaptive momentum estimation (Adam)³⁰ optimizer was used for training the network.

Figure 2.

Input to the neural network. The cortical data was mapped to squares and subcortical data was vectorized according to the grayordinate convention.

Figure 3.

The Neural Network model is a combination of a convolutional and a fully connected network. Geometric and functional features were mapped to the flat maps for the two cortical hemispheres and were input to a VGG style CNN, whereas a fully connected neural network was used for data from the subcortical regions.

The training was performed on 169 subjects. For the Adam optimizer, a batch size of 10 was used with 8 subjects for computing the update to the weights and 2 subjects used for computing the cross-validation error. The weights were updated when the cross-validation error decreased. Small batch size was used in order to avoid over-fitting. 100 Epochs were performed. The training was done on a workstation computer (Intel Xeon E5-2650 v3). GPU acceleration was not used for this dataset. The training took about 10 hours. Once the model was trained, it was used to predict the three cognitive scores on an independent test dataset comprised of the remaining 90 subjects.

3. RESULTS

To analyze the performance of the network, we computed the correlation between the predicted values of the three cognitive indices and their actual values for the 90 test subjects. This computation was performed for networks trained on: (i) functional features computed from rfMRI data only (C = 21 channels), (ii) geometric features computed from the shape of the cortex (C = 15 channels), and (iii) the combination of both rfMRI and shape features (C = 36 channels). The results are summarized in Figure 4. We observed a substantial correlation between predicted and actual values of the indices tested: Performance IQ: 0.47; Verbal IQ: 0.41, ADHD: 0.57. The results show that the combination of fMRI and geometric features result in a substantially improved predictability of these indices relative to either structure or function alone.

Figure 4.

The table on the left shows the correlation between actual and predicted cognitive scores for prediction based on shape features only, rfMRI features only, and the combination of the two. The plot on the right shows scatter plot of actual and predicted ADHD index using the combination of shape and rfMRI features, for the test population. These results were generated based on the training with 169 and testing on the remaining 90 independent subjects.

4. DISCUSSION AND CONCLUSION

We presented a deep learning method to predict the cognitive performance scores from the structural and functional imaging data derived from T1-weighted MRI and rfMRI. We also presented a method for representing this data at high resolution so that it can be used as input to a CNN. Most existing methods use graphical or geometric CNN for analyzing cortical data.³¹ Comparing the values to those from network trained on functional-only and structural-only data, we see that rfMRI is more informative than structural fMRI, but the two modalities are highly complementary in terms of predicting these indices.

One limitation of the presented methodology is that the flat mapping of the cortex to unit squares can introduce a large amount of metric distortion. Weighting the convolution kernels to account for this distortion will allow the shared kernels to perform equivalently with respect to shape throughout the cortex. The presented method does not use the metric of the surface for modifying the convolution kernels. We intend to explore this possibility in the future.