Discovering individual fingerprints in resting‐state functional connectivity using deep neural networks

Abstract

Non‐negligible idiosyncrasy due to interindividual differences is an ongoing issue in resting‐state functional MRI (rfMRI) analysis. We show that a deep neural network (DNN) can be employed for individual identification by learning important features from the time‐varying functional connectivity (FC) of rfMRI in the Human Connectome Project. We employed the trained DNN to identify individuals from an independent dataset acquired at our institution. The results revealed that the DNN could successfully identify 300 individuals with an error rate of 2.9% using 15 s time‐window and 870 individuals with an error rate of 6.7%. A trained DNN with nonlinear hidden layers led to the proposal of the “fingerprint of FC” (fpFC) as representative edges of individual FC. The fpFCs for individuals exhibited commonly important and individual‐specific edges across time‐window lengths (from 5 min to 15 s). Furthermore, the utility of our model for another group of subjects was validated, supporting the feasibility of our technique in the context of transfer learning. In conclusion, our study offers an insight into the discovery of the intrinsic mode of the human brain using whole‐brain resting‐state FC and DNNs.

By using deep neural networks (DNNs), reliable and robust fingerprints of resting‐state functional connectivity for individuals were found. The trained DNN showed its efficacy in the identification of individuals from an independent dataset via transfer learning.

Practitioner Points.

• Deep neural network (DNN) identified individuals using resting‐state functional MRI (rfMRI).

• The fingerprint of functional connectivity (fpFC) for an individual was obtained from the trained DNN.

• Important edges in the fpFC were shared across individuals for varying time‐window lengths in rfMRI.

• The number of important edges in the fpFC showed a gradual change across the time‐window lengths.

• The trained DNN was applied to the identification of individuals from an independent dataset.

1. INTRODUCTION

Functional connectivity (FC) from the functional MRI (fMRI) modality has been widely used to investigate positively or negatively synchronized neural activations across the whole brain (Biswal et al., 1995; Fox et al., 2005; van den Heuvel & Hulshoff Pol, 2010). Many studies have consistently reported the existence of various functional networks (FNs) commonly found across subjects (Noble et al., 2019). However, despite these similarities, there remains significant heterogeneity across individuals in terms of FC (Finn et al., 2015; Gordon, Laumann, Adeyemo, & Petersen, 2017; Gordon, Laumann, Gilmore, et al., 2017; Gratton et al., 2018; Vanderwal et al., 2017). Resting‐state fMRI (rfMRI) is dominated by spontaneous activity without the involvement of a particular stimulus or task. Because resting‐state FC is thought to characterize intrinsic functions (Biswal et al., 1995), it has been frequently employed in various studies. For example, previous reports have identified individuals using their FC information obtained from rfMRI (Cai et al., 2020; Chen & Hu, 2018; Demeter et al., 2020; Finn et al., 2015; Finn et al., 2017; Horien et al., 2019; Pallarés et al., 2018; Sarar et al., 2021). In this context, there have been attempts to uncover the distinguishing interindividual differences of FCs. In previous works, the representative profile of an individual has often been termed “fingerprint” (Amico & Goñi, 2018; Cai et al., 2020; Demeter et al., 2020; Finn et al., 2015; Liu et al., 2018; Van De Ville et al., 2021).

Time‐varying FC (tvFC), also known as dynamic FC, has been conventionally estimated using one of two approaches: (a) calculating the changes in FC across regions directly using the observed BOLD signals and (b) modeling underlying neural dynamics such as by using neural mass and neural field models (Lurie et al., 2020). The ordinary sliding window approach refers to calculating a series of FC patterns using a fixed time window of seconds (s) or minutes (min) at shifting time points (Hindriks et al., 2016; Hutchison et al., 2013). There are alternative approaches to the sliding window approach with a fixed window length, such as adaptive windowing, Hidden Markov models, co‐activation pattern analysis, and clustering (Yaesoubi et al., 2018). Numerous studies have employed static FC to extract a single FC pattern from time series data spanning the entire scan duration. However, other studies have explored the application of brief window lengths to extract a series of tvFC patterns, revealing distinct temporal dynamics unique to each individual (Fong et al., 2019; Preti et al., 2017; Savva et al., 2019). In the present study, we examined whether the tvFC utilizing short window lengths could exhibit reliable individual‐specific information, comparable to the expectations of static FCs derived from extended window lengths, while concurrently accommodating the temporal variation inherent in tvFC patterns. A very short window is limited in producing reliable information due to the lower signal‐to‐noise ratio arising from confounding factors such as physiological artifacts and scanner hardware noise disrupting the measurement of the synchrony of FNs within the brain (Murphy et al., 2013; Power et al., 2020). The use of tvFC with a short window can thus be challenging, but it remains potentially useful. Therefore, the development of computational models that can extract robust FC features from noisy tvFC patterns is required to enhance the utility of tvFC. Although previous studies have investigated the effects of various time window lengths on tvFC, consensus on the optimal window length for significant findings remains elusive (Gonzalez‐Castillo et al., 2014; Preti et al., 2017). This lack of definitive evidence prompted our investigation into the feasibility of individual identification across various window lengths, from less than a minute to several minutes. Prior attempts to distinguish individuals using FC features faced limitations, with error rates of less than 1% being achieved using static FC or a confined subject pool (Cai et al., 2020; Demeter et al., 2020). The pursuit of minimal error rates in the identification of a large cohort using tvFC patterns computed from brief time‐windows has presented substantial challenges (Chen & Hu, 2018; Sarar et al., 2021; Van De Ville et al., 2021). To the best of our knowledge, no study has yet been able to identify hundreds of participants with high accuracy using time‐windows under 1 min, nor delineated common or unique edges across various window lengths.

Deep neural network (DNN) models have been widely employed in classification tasks because of their superior performance and ability to organize hierarchical feature representations (Abrol et al., 2021; Bengio et al., 2013; Plis et al., 2014). Despite their efficacy, however, overfitting has been a long‐standing issue for DNNs due to the large number of tunable weight parameters, which can easily compromise generalization performance (Baum & Haussler, 1989; MacKay, 1992; Neal, 1996; Schmidhuber, 2015). To prevent overfitting and to enhance the generalization performance of a DNN, a regularization penalty using weight parameters is frequently added to the cost function of the DNN, such as an L1‐norm term (e.g., LASSO), an L2‐norm term (e.g., ridge regression), or both (e.g., elastic net) (Zou & Hastie, 2005). An explicit weight sparsity control scheme for a DNN combines the advantages of both L1 (which helps to select essential features from input patterns) and L2 regularization (which helps to stabilize and collect important features) (Kim et al., 2016). DNN models with explicit weight sparsity control have exhibited superior performance for both classification and regression problems due to their excellent hierarchical feature extraction capability, which is crucial for high‐dimensional and noisy whole‐brain fMRI data (Jang et al., 2017; Kim et al., 2016; Kim, Bandettini, & Lee, 2019; Kim & Lee, 2017). Recent efforts have been directed at extracting individual‐specific information from FC patterns using DNN (Cai et al., 2020; Chen & Hu, 2018; Sarar et al., 2021) and machine learning (ML) approaches (Vergara et al., 2018). Both DNN and ML employ model‐based approaches capable of identifying crucial features for the model's objective through learning from the input. However, DNNs not only demonstrate superior feature extraction capabilities but also facilitate the derivation of hidden representations through the input–output mapping process, which promotes transfer learning. This capability extends beyond the more limited scope of shallow ML models that are confined to learning decision boundary approaches (Abrol et al., 2021).

In the present study, we investigate the possibility of individual identification across different days using a weight‐sparsity‐controlled DNN as the computational model and tvFC as the input. From the trained DNN model, we report representative FC edges from the tvFC patterns of the human brain and refer to these as the fingerprint of FC (fpFC). The performance of the DNN is systematically evaluated using rfMRI data available from the Human Connectome Project (HCP) and compared with that of alternative linear classifiers. We also investigate the feasibility of using the trained DNN to classify individuals using rfMRI data from our institute. We hypothesize that (1) DNN identification performance will be superior to that of shallow ML models, (2) individuals have a distinct tvFC fingerprint during a resting state even for shorter window lengths, and (3) important FC edges for individual identification are robust across various window lengths despite the presence of a moderate level of variability. This study presents that the DNN model can distinguish hundreds of individuals, even when FC is measured for a very short period of time (only a fraction of a minute). Moreover, the trained model is capable of providing a representative fingerprint‐like feature for each individual (i.e., fpFC). These fpFCs could serve as individual “neuromarkers” (Finn et al., 2015), offering the potential to associate with and predict behavioral phenotypes and/or dysfunction toward the development of prognosis prediction and personalized treatment options (Kong et al., 2021; Van De Ville et al., 2021).

2. MATERIALS AND METHODS

2.1. Overview

Figure 1 presents a schematic diagram of our DNN model for individual identification from rfMRI data of the HCP dataset (Van Essen et al., 2013). The identity of an individual was predicted in the output layer based on the input tvFC patterns. The trained DNN was validated and tested using tvFC patterns acquired on another day.

FIGURE 1.

Individual identification using resting‐state tvFC and a DNN. (a) DNN classifier to identify individuals. The number of input dimensions for the DNN is 62,835 (i.e., FC edges from paired ROIs across all 355 ROIs). The number of output units corresponds to the number of individuals to be identified. The Softmax in the output layer was used to obtain the probability for each individual. The fpFCs were computed from the trained parameters of the DNN. (b) The tvFC patterns from one of the two visits were used for training, and the tvFC patterns from the other visit were used to validate and test the trained DNN model. (c) TvFC patterns were calculated from four rfMRI runs in the HCP dataset obtained from two visits. DNN, deep neural network; FC, functional connectivity; HCP, Human Connectome Project; rfMRI, resting‐state fMRI; ROIs, regions‐of‐interest; tvFC, time‐varying FC.

2.2. fMRI data and preprocessing

We used rfMRI data available from the HCP S1200 dataset (healthy young adults; age = 22–35) (Van Essen et al., 2013). The rfMRI data were acquired on a customized Siemens 3 T Connectome Skyra with a standard 32‐channel head coil at Washington University (Elam et al., 2021). A total of 1200 volumes (approximately 14.4 min) were obtained from each rfMRI run (TR/TE = 720/33.1 ms, voxel size = 2.0 mm isotropic, slice thickness = 2.0 mm without a gap, flip angle [FA] = 52°, field‐of‐view [FoV] = 208 × 180 mm²). Subjects were instructed to open their eyes and gaze at a crosshair while scanning. Each subject visited twice (on Days 1 and 2), and went through two runs of rfMRI where echo‐planar imaging (EPI) volumes were acquired in two different phase‐encoding directions (i.e., right‐to‐left [R‐L] and left‐to‐right [L‐R]).

Several preprocessed versions of the raw rfMRI data are readily available within the HCP S1200 dataset (Glasser et al., 2013; Glasser et al., 2016; Griffanti et al., 2014; Smith et al., 2013). We used the preprocessed data from the HCP pipeline scripts v3.13.2. with MSMAll DeDrifting and Resampling based on MSMAll registration (https://github.com/Washington-University/HCPpipelines/releases/tag/v3.13.2). More specifically, the BOLD time series at cortical vertices has been registered to a template using MSMAll pipelines and optionally denoised using ICA‐FIX algorithms (https://github.com/Washington-University/HCPpipelines/tree/master/MSMAll), which is effective for ruling out connectivity due to spurious signals (Xifra‐Porxas et al., 2021). In addition, the BOLD time series of rfMRI data underwent band‐pass filtering in the range of 0.01–0.1 Hz. A total of 870 individuals out of approximately 1200 had four intact rfMRI runs with 1200 volumes per run and their rfMRI volume‐series files were available for several preprocessing strategies (i.e., MSMAll/MSMSulc registration and with/without ICA‐FIX denoising). The MSMSulc only used myelin maps (cortical folding patterns) to guide the registration process, whereas the MSMAll also incorporated rfMRI. The primary focus of our analysis was on datasets processed using MSMAll; however, we conducted comparative evaluations on datasets using MSMSulc as well.

2.3. Regions‐of‐interest and functional networks

The 1200‐volume rfMRI data were registered to a template in the gray ordinate coordinate system, in which there are 59,412 vertices across the bilateral hemispheres. The multimodal parcellation (MMP) map was used to acquire the time series for 360 cortical regions‐of‐interest (ROIs) (Glasser et al., 2016). We grouped our 360 ROIs based on seven cortical FNs (Thomas Yeo et al., 2011), which include the visual network (VN), somatomotor network (SMN), dorsal attention network (DAN), ventral attention network (VAN), limbic network (LN), frontoparietal network (FPN), and default‐mode network (DMN) (Figure S1). Because the seven FNs had been defined using voxels, we projected them onto vertices then assigned FN membership for each ROI based on the most extensive overlap (i.e., a winner‐takes‐it‐all approach). Then, 5 out of 360 ROIs, residing in the medial wall near the cingulate cortex, did not overlap with any of the 7 FNs. Thus, 355 ROIs in total were assigned to the seven FNs.

In addition, we compared the identification performance using brain templates with fewer brain regions than present in the Glasser 360 atlas. To this end, we chose Schaefer's 100 and 200 ROIs defined using a gradient‐weighted Markov Random Field (gwMRF) model (Schaefer et al., 2018). All ROIs were distributed among the seven FNs (https://github.com/ThomasYeoLab/CBIG/tree/master/stable_projects/brain_parcellation/Schaefer2018_LocalGlobal/Parcellations/HCP/fslr32k/cifti) (Figure S2).

2.4. Estimation of tvFC patterns

We averaged the BOLD time series in each of 355 ROIs across the vertices, yielding a 355 × 1200 matrix for each rfMRI run. The tvFC patterns were obtained based on the sliding window approach. We calculated the Pearson's correlation coefficients between the time series for the 355 ROIs within each time window (Figure 1b,c). A vector of the upper diagonal edges of the 355 × 355 correlation matrix was 62,835 dimensional (cf. 4950 and 19,900 for Schaefer's 100 and 200 ROIs, respectively), which was Fisher's r‐to‐z transformed and pseudo z‐scored. The window slid forward from the first volume with no overlap with the previous window to prevent dependency between adjacent tvFC patterns. We obtained tvFC patterns for each of several window lengths ranging from seconds to minutes (i.e., 15 s, 30 s, 1 min, 3 min, and 5 min). Considering the passband for the band‐pass filtering (0.01–0.1 Hz, or equivalently, 10‐s to 100‐s cycles), we set the shortest window length as 15 s. The number of tvFC patterns for each rfMRI run (14.4 min) was 57, 28, 14, 4, and 2 for the 15‐s, 30‐s, 1‐min, 3‐min, and 5‐min windows, respectively. For each window length, corresponding tvFC patterns from all the four rfMRI runs per subject were used for training and the validation/testing of prediction models.

2.5. Individual identification

A fully connected DNN was employed to identify individuals (i.e., subjects) using the input tvFC patterns, where each of the output units from the DNN was assigned for each individual. Each hidden layer consisted of 2000 units. Thus, the number of units per layer was 62,835 for the input layer, 2000 for each of the hidden layers 1 and 2, and N for the output layer, where N is the target number of individuals to identify. The DNN was trained to classify an individual's identity as a target label ($y_i$). This multi‐class classification was implemented via the Softmax layer in the output (${\widehat{y}}_i$). The cost function for the DNN used to train the weight parameters was as follows:

$$J\left(x,w\right)=-{\sum }_i{y}_i\log \left({\widehat{y}}_i\right)+{\sum }_l{\beta }_l{‖w_l‖}_1+\gamma {\sum }_l{‖w_l‖}_2$$ (1)

where ${\widehat{y}}_i=\mathit{\text{softmax}}\left(z_i\right)$, $z_i$ is the output of the last layer corresponding to input $x_i$, ${\beta }_l$ is the L1‐norm parameter for the weights in the lth layer of the DNN ($w_l$), and $\gamma$ is the L2‐norm parameter.

The weight sparsity levels between the DNN layers were explicitly controlled via the strength of ${\beta }_l$, and the optimal sparsity level was identified (see the supplementary materials, “Weight Sparsity Optimization” for more details including hyperparameter tuning). The $\gamma$ was fixed as 10⁻⁸. The weight sparsity‐controlled DNN has shown better performance in classification/regression tasks than elastic net regularization (Kim, Bandettini, & Lee, 2019). This is because of the fine‐grained optimization of sparsity level using L1‐norm penalty via adaptive ${\beta }_l$ during the training phase, compared to the optimization of the L1‐norm penalty term without an explicit weight sparsity control. The optimizer was stochastic gradient descent with an initial learning rate of 0.03 and a momentum of 0.9. The mini‐batch size varied from 13 to 1102 depending on the total number of individuals and the number of samples available for each individual (varying across window lengths). The activation function in each hidden layer was a hyperbolic tangent (tanh), and batch normalization was applied to the hidden layers. A rectified linear unit did not outperform tanh in our study. The DNN classifier was implemented using the deep learning library TensorFlow 1.15 (https://www.tensorflow.org/) in Python 3.6. The code and sample data are publicly available at our GitHub repository (https://github.com/bsplku/fingerprintFCdnn). Several configurations for the target number of individuals for identification were tested (i.e., 100, 300, 600, and 870 out of the 870 individuals from HCP S1200 with 4 rfMRI runs). This target number corresponded to the number of output nodes (N) of the DNN.

The training set consisted of the tvFC patterns calculated from the EPI volumes in the two rfMRI runs with R‐L and L‐R phase‐encoding on Day 1. The EPI volumes from Day 2 were used as the validation (rfMRI run with R‐L phase‐encoding) and test (rfMRI run with L‐R phase‐encoding) sets, respectively. Classification performance was evaluated for the training, validation, and test set, by calculating the error rate using all tvFC patterns/samples across all the subjects in each set. We additionally ran a slightly modified version of train‐and‐evaluate: training DNNs with all window lengths but not a 15‐s window and validation/test with a 15‐s window. It was to evaluate whether DNNs trained with varied window lengths were valid for noisy tvFCs from the shortest window as well.

Because physiological noise and head motion alter the BOLD time series and subsequently the FC patterns, the possibility that these confounding artifacts contributed to individual identification was investigated. Specifically, the identification performance using ICA‐FIX‐based time series denoising was evaluated. In addition, we monitored head motion on Day 1 using the root mean square (RMS) of frame‐wise displacement available in the “Movement_RelativeRMS” file in the HCP dataset. Then, the identification performance was compared at a subject level between individuals with high and low head motions.

Pattern matching (PM) between the training samples and the validation/test samples was conducted based on Pearson's correlation coefficients in MATLAB 2017b. Logistic regression (LR) and linear support vector machine (SVM) were tested as shallow linear ML models as a comparison with our DNN model. The LR (the “LogisticRegression” class) and SVM (the “LinearSVC” class) algorithms implemented in the Scikit‐learn library (version 0.22.1; Python 3.6) were used with an L2 penalty. For LR, we chose the “newton‐cg” solver from among the five optional solvers available in Scikit‐learn because of its superior performance for large HCP datasets. The error rate for LR was investigated for both the multinomial scheme (to allow a comparison with the DNN) and one‐versus‐rest (OvR) schemes (to allow a comparison with SVM, whose algorithm only supports OvR). The L2 penalty term was optimized using a grid search (C ∈ {10², 5 × 10², 10³, 3 × 10³, 5 × 10³, 10⁴, 3 × 10⁴, 5 × 10⁴} for LR; C ∈ {10⁻⁵, 10⁻⁴, 5 × 10⁻⁴, 10⁻³, 5 × 10⁻³, 10⁻², 5 × 10⁻², 10⁻¹, 5 × 10⁻¹, 1, 5, 10, 50, 10², 5 × 10², 10³, 5 × 10³, 10⁴} for SVM). We performed a nonparametric test, specifically the Wilcoxon signed‐rank test, to compare the error rates of DNN with the error rate of LR, due to the small sample size and non‐normal distribution. A one‐tailed test was selected, positing the null hypothesis that DNN's performance would not surpass that of LR, against the alternative hypothesis that DNN would outperform LR. To determine whether error rates varied with the number of layers, we performed the Wilcoxon rank‐sum test, a nonparametric analogue to the two‐sample t test. Furthermore, we conducted a one‐sample t test to assess the difference in error rates across all subjects compared with the chance level and a paired‐sample t test to compare subject‐wise error rates between two DNN models.

2.6. Fingerprint of FC

We extracted an FC signature for individual identification that is hidden within the tvFC patterns from a trained DNN model. To this end, the weight feature vectors $\mathit{WF}$ (62,835 × N) were estimated by multiplying the weight parameter matrices $w_l$ across all layers from the input layer to the output layers (Equation 2) (Erhan et al., 2009; Jang et al., 2017; Kim et al., 2016; Kim, Bandettini, & Lee, 2019):

$$\mathit{WF}={\prod }_l{w}_l$$ (2)

The z‐scored vector $W{F}_n$ (62,835 × 1) yielded an fpFC for an individual, where n indicates an index of output unit corresponding to the individual. Based on both qualitative and quantitative criteria, we labeled the edges within the fpFCs with four designated categories: (a) reliable edge, (b) distinct edge, (c) important edge, and (d) robust edge (Table 1). For the quantitative definition of an edge based on fpFCs, we calculated the standard deviation (STD) across the target individuals and an intra‐class correlation coefficient (ICC; two‐way random effects with absolute agreement) (McGraw & Wong, 1996) across the target individuals and training repetitions using MATLAB. Only the edges whose ICCs were greater than 0.6 (Cicchetti, 1994; Landis & Koch, 1977) were considered to be (a) reliable and preserved in the subsequent analysis to determine the distinct edges. The heterogeneity of the fpFCs across the individuals was also evaluated. The STD was calculated for each edge of an average fpFC from the five repetitions across individuals. The upper 98th percentile of the STD was reported as a (b) distinct edge. The edges that met the thresholds for both reliability (i.e., ICC >0.6) and distinctiveness (i.e., STD > 98th percentile) were defined as (c) important edges. Important edges that were common across the five window lengths were obtained and classified as (d) robust edges which are invariant to window length. The number of important edges for each FN pair was compared between window lengths. Unless otherwise stated, this ICC and STD analysis was conducted using the fpFCs obtained from two‐hidden‐layer DNN models trained to identify 300 individuals from the HCP. To evaluate the stability of error rates and the significance of FC edges, we repeated the training of DNN models an additional 20 times, culminating in a total of 25 iterations. FC edges were visualized using circular plots and a 3D brain (https://immersive.erc.monash.edu/neuromarvl/). Generating an activation maximization map $\mathit{AM}$ can be an alternative method to interpret the trained DNN model, rather than weight features. To this end, input FC edges that maximized each of the output units were estimated using a gradient ascent approach via back‐propagation (Equation 3) (Erhan et al., 2009):

$$\mathit{AM}=\underset{x}{\text{argmax}}\mathit{log}p\left(y|x\right)-\gamma {‖x‖}_2$$ (3)

fpFCs were similarly obtained by z‐scoring a vector at an output unit.

TABLE 1.

Glossary for connectivity edges defined from fpFCs.

Term	Qualitative definition	Quantitative definition
Reliable edge	Consistent across multiple training instances with random initialization	High ICC (>0.6)
Distinct edge	Different across subjects	High STD (>98th percentile)
Important edge	Consistent and different across subjects	High ICC and high STD
Robust edge	Consistent and different across subjects for all window lengths	High ICC and high STD for all window lengths

Abbreviations: fpFC, fingerprint of functional connectivity; ICC, intra‐class correlation coefficient; STD, standard deviation.

2.7. Reproducibility of the fpFC for various training configurations

An individual fpFC should be consistent between visits (i.e., sessions; Days 1 and 2) and not be biased toward a particular group of individuals. We thus evaluated the specificity of the obtained fpFC for six configurations, with distinct groups of individuals used for identification and the training and validation/test data shuffled. Specifically, we created three groups of individuals within the training dataset for the DNN model, each encompassing one common target individual (Figure S3a): (I) an initial group with 300 individuals including the target individual, (II) a second group with 300 individuals including 299 new individuals and the target individual, and (III) a third group that adds another 300 new individuals to the second group (i.e., a total of 600 individuals). We obtained six independent training outcomes by employing the three distinct groups while alternating the training and validation/test sets (i.e., Day 1 data for training and Day 2 data for validation/testing; Day 2 data for training and Day 1 data for validation/testing). Similarity scores across the fpFCs were measured using Pearson's correlation coefficients for the configurations. For each pair of configurations, the similarity of the target individual's fpFCs was compared with the similarities of all other individuals' fpFCs.

2.8. Genetic influence on the fpFC

Due to previous reports indicating that genetic relatedness is reflected in the fpFC (Finn et al., 2015; Liu et al., 2019; Mueller et al., 2013), we were motivated to examine whether our reported fpFC originated in part from genotypical characteristics. We divided individuals from the HCP into four groups according to their genetic relatedness: (a) monozygotic twins (MZ), (b) dizygotic twins (DZ), (c) siblings, and (d) others. The similarity of the fpFC was calculated for paired individuals within each group and compared between groups. More specifically, first, we obtained the average fpFC for each individual across the five window lengths using mean fpFC from the five trained DNNs to classify 300 individuals with random initial weights for each window length. We stratified the 300 individuals into pairs of individuals based on their genetic relatedness: MZ, who shared identical genes, DZ, siblings, and others. The averaged fpFCs were compared across these stratified groups of individuals. The correlation coefficients for the average fpFCs were Fisher's r‐to‐z transformed. The z‐scored similarity of the representative fpFCs was compared across the groups using one‐way ANOVA followed by a post hoc two‐sample t test. The resulting p values were Bonferroni‐corrected.

2.9. Generalization of the trained DNN to an independent dataset via transfer learning

In this study, we evaluated the application of a DNN, which had been trained on HCP data in the processing of independent datasets through the extrapolation of learned characteristics. For this purpose, we conducted a similar rfMRI experiment at our institute following the HCP protocol using a 3‐T MRI scanner (MAGNETOM Tim Trio; Siemens, Erlangen, Germany) and a 12‐channel head coil (to display cross‐fixation for subjects with their eyes open during rfMRI runs via MR‐compatible visual binocular goggles). A multiband (MB) gradient‐echo EPI pulse sequence developed by the Center for Magnetic Resonance Research (Department of Radiology, University of Minnesota) was used to acquire the rfMRI data at our institute. The specific imaging parameters were as follows: MB factor = 2; Generalized Auto‐Calibrating Partial Parallel Acquisitions (GRAPPA) on; TE = 30 ms; FA = 71°; FoV = 192 × 192 mm²; 50 axial slices without a gap; TR = 1.44 s (twice that of the HCP [0.72 s] due to the lower parallel imaging factor to cover the whole brain); and voxel size = 3 × 3 × 3 mm³ (Kim, Tegethoff, et al., 2019; Tozzi et al., 2020). In addition, we collected a gradient‐echo field map to correct EPI distortion as required from the HCP pipelines (https://github.com/Washington-University/HCPpipelines).

The overall study protocol was approved by the Institutional Review Board (IRB) at Korea University. The participants submitted written consent forms and were compensated as outlined in the IRB documents. Fifteen healthy young adults participated in the experiment (age = 23–34, seven females). Fourteen of these subjects (all except one female) made two visits on consecutive days. They were instructed to gaze at a white cross‐fixation on a black screen during the rfMRI run. Their eyes were monitored via eye trackers attached to the goggles throughout the rfMRI run (NordicNeuroLab; nordicneurolab.com/). Two rfMRI runs with R‐L and L‐R phase‐encoding and a T1‐weighted image were acquired on the first day, while two rfMRI runs with R‐L/L‐R phase‐encoding and T2‐weighted images were acquired on the second day. The rfMRI runs were preprocessed using the HCP pipelines (Glasser et al., 2013; Glasser et al., 2016; Griffanti et al., 2014). Subsequently, the tvFC patterns were obtained from the preprocessed rfMRI BOLD time series described in Section 2.4. The number of time points used for FC calculation with the same window length was half that of the HCP because the TR for our dataset was twice as long.

A pretrained DNN for the 870 individuals from the HCP using tvFC patterns from a 15‐s window was restored for transfer learning. The TensorFlow variables of the pretrained DNN, such as the weights, biases, and batch‐normalization‐related parameters from the first to the last hidden layers, were restored. A new output classification layer was created with 314 units to accommodate the 14 new individuals from our institute and the 300 individuals from the HCP, and randomly initialized. During the transfer learning phase, the DNN model was trained under either of two conditions: (1) freezing parameters from the first to the last hidden layers and fine‐tuning only the classification layer, and (2) fine‐tuning all layers. A DNN model trained using randomly initialized weights across all layers was evaluated as a baseline model.

3. RESULTS

3.1. Individuals are more clearly distinguished using a DNN than using shallow linear models

Figure 2 shows the error rates across various scenarios. For 300 individual identification cases, the longer the time window for the tvFC patterns, the lower the error rate, particularly for PM (Figures 2a and S4). Model‐based approaches, including linear SVM, LR, and the DNN exhibited a remarkably lower error rate compared to PM. Notably, there was a substantial reduction in the error rates of LR and DNN for a 30‐s window (e.g., 83.2% for PM, 0.4% for LR, and 0.2% for the DNN) compared with a 15‐s window (e.g., 91.4% for PM, 5.1% for LR, and 2.9% for the DNN). For all window lengths, two‐hidden‐layer DNNs had lower error rates than LR (DNN: 2.9, 0.2, 0.1, 0.1, and 0.2% for 15 s, 30 s, 1 min, 3 min, and 5 min‐windows, respectively; LR: 5.1, 0.4, 0.2, 0.3, and 0.3%; one‐tailed Wilcoxon signed‐rank test, N = 5, W = 0, p = .0312). The performance of the DNNs remained consistently superior through an additional 20 repetitions of the training process for the DNNs (DNN: 3.0, 0.2, 0.1, 0.1, and 0.1%; N = 25, W = 0, p = 3.0 × 10⁻⁸). We could not obtain identification results for the 15‐s window from SVM because there was an overflow error due to the large sample size. An alternative approach is to use libsvm‐based SVC instead of LinearSVC; however, it produced an inferior performance and was computationally expensive. For example, the test error rate for a 15‐s window and 300 individuals was 39.3% using the trained libsvm‐based SVC model and took about a month to compute using our system (two Intel Xeon Gold 5120 CPUs). The error rate generally increased as the number of individuals increased (Figure 2b). The DNN was consistently the best performing model, with an error rate of 6.7% for a 15‐s window with 870 individuals (cf. 9.9% for LR).

FIGURE 2.

Test error rates for the individual identification of Human Connectome Project (HCP) subjects for various scenarios. (a) Error rate for 300 individual identification scenarios for various window lengths. The error rate for LR was consistently higher using the OvR scheme compared to the multinomial scheme. We conducted five repetitions of training and testing for the DNN with random initialization at each window length. (b) Error rate for various numbers of individuals to be identified using 15‐s window‐based tvFC, time‐varying functional connectivity (tvFC). Multinomial target output was used for LR. A DNN identifying 300 individuals was repeated five times, and a DNN identifying 100 or 600 individuals was run with three repetitions. DNN, deep neural network; LR, logistic regression; OvR, one‐versus‐rest; PM, pattern matching; SVM, support vector machine.

The DNN models trained using tvFC patterns from longer window lengths than 15 s were tested using tvFC patterns from a 15 s window. The error rates of 300 individual identification models from 30 s, 1 min, 3 min, and 5 min windows were 3.7, 6.5, 16.5, and 52.0%, respectively. These rates are notably low given that the chance level is 1–1/300 = 99.7% (one‐sample t test, df = 299, p = .0). The DNN trained on the aggregate of the four longer windows (i.e., 30 s, 1 min, 3 min, and 5 min) exhibited a test error rate of 4.0% on the 15 s window, which was higher than 3.7% from the DNN trained solely on the 30 s window (paired‐sample t test, df = 299, T = 2.2, p = .028). This error rate was also higher than the 2.9% from the DNN trained using the FC patterns from the 15 s window (paired‐sample t test, df = 299, T = 6.3, p = 8.6 × 10⁻¹⁰).

Registration by MSMSulc instead of MSMAll or time series denoising using ICA‐FIX improved the classification performance (Figure S5). The error rate for the 21 individuals with high head motion (RMS [mm] > .14; Finn et al., 2015) was 5.8% ± 6.6, which was greater than the error rate of 2.7% ± 4.0 for the remaining 279 individuals (two‐sample t test, df = 298, T = −3.2, p = .0016). An adaptation of an atlas with fewer ROIs led to a higher error rate (Figure S6). Error rates of DNN and LR using Schaefer's 100 and 200 ROIs were still substantially lower than the PM. However, DNN did not outperform LR for the 5‐min window using 200 ROIs, or 3‐ and 5‐min windows using 100 ROIs.

3.2. The weight‐sparsity‐optimized DNN is superior to alternative models

For tvFCs with a 15‐s window, the DNN model trained with optimal weight sparsity levels performed better when there were two hidden layers (validation/test error rate = 4.1/2.9%) compared to one hidden layer (5.3/3.7%; Wilcoxon rank‐sum test, N = 5/5, U = 25/25, p = .0079/.0079). While the test set benefitted significantly from the two hidden layers, the improvement in the validation set was marginal when compared to the three‐layer setup (4.2/3.1%; Wilcoxon rank‐sum test, N = 5/5, U = 21/25, p = .0952/.0079) hidden layers (Figure S7). The test error rate for the two‐hidden‐layer DNN (2.9%) was lower than that of the LR (5.1%; one‐tailed Wilcoxon signed‐rank test, N = 5, W = 0, p = .0312), representing an improvement of 43.1%. The optimal weight sparsity for each hidden layer differed according to the depth of the DNN. The results for optimal weight sparsity with varying numbers of hidden layers are described in the “Optimal Levels of Weight Sparsity with Varying Numbers of Hidden Layers” section of the Supplementary Materials.

3.3. FpFC characterizes individual connectivity profiles and suggests robust edges

Overall, the edges with high ICCs exhibited strong variability across individuals (Figures 3a and S8). We primarily found window‐length invariant robust edges in the DMN and FPN, many of which are located in the frontal lobe (Figure 3b; Table 2). Notably, the bilateral dorsolateral prefrontal cortex (DLPFC; i.e., area 9 anterior) was included in the top three most critical regions in the FPN. In addition, intra‐FN and intra‐hemispheric connections were dominant over inter‐FN and inter‐hemispheric connections. Robust edges accounted for almost half of the important edges for each window length; in particular, 47.3, 47.3, 47.3, 52.1, and 48.0% of the important edges identified for the 15‐s, 30‐s, 1‐min, 3‐min, and 5‐min windows, respectively, were classified as robust. Across different numbers of ROIs, we generally found noticeable engagement of intra‐hemispheric connections in the frontal lobe for individual identification (Figures 3b, S9a, and S10a). There was a larger involvement of the occipital lobe using 100 and 200 ROIs from Schaefer's atlas than 355 ROIs from Glasser's atlas. The edges in intra‐DMN and DMN‐FPN were consistently significant (Figures 3c, S9b, and S10b). However, the edges in intra‐VN, VN‐FPN, SMN‐DAN, or VAN‐DMN were more pronounced for 100 ROIs and 200 ROIs compared to 355 ROIs. On the other hand, the significance of edges in intra‐DAN, intra‐FPN, VN‐DAN, DAN‐VAN, DAN‐FPN, LN‐FPN, LN‐DMN, or FPN‐DMN was reduced. The fpFC from activation maximization maps resulted in patterns that were markedly similar to our weight feature representation based on the multiplication of weight matrices (Figure S1a,b). The weight feature vectors suggested 595 robust edges across window lengths, while activation maximization suggested 591 robust edges. A total of 582 robust edges were identical between these estimates. Nonetheless, some edges existed that were found by weight feature vectors but not by activation maximization, or vice versa (Figure S11c). Most of the edges significant only for weight feature vectors were connected to the ROIs in the frontal cortex, predominantly in FPN and DMN. In contrast, activation maximization not only revealed the important edges of the frontal lobe but also indicated associations with other brain areas. For example, there were edges in FPN‐VAN, including a connection between the frontal and the temporal lobe.

FIGURE 3.

Informative edges in the fpFC for individual identification. (a) ICC and STD for each edge in the fpFCs across five window lengths. (b) Robust edges (c) The number of important edges from intra‐ and inter‐FNs. In each plot, the brighter bars at the bottom indicate the number of robust edges. The darker bars above represent important but not robust edges for each window length. The plots on the right present the number and percentage of important edges for the corresponding FN pair between window lengths. Please see Table 1 for the details on the definition of robust and important edges. The fpFC patterns were obtained from DNN models for the identification of 300 individuals. DAN, dorsal attention network; DMN, default‐mode network; FN, functional network; fpFC, fingerprint of functional connectivity; FPN, frontoparietal network; ICC, intra‐class correlation coefficient; LN, limbic network; SMN, somatomotor network; STD, standard deviation; VAN, ventral attention network; VN, visual network.

TABLE 2.

List of the 30 most frequently involved ROIs in the robust edges of the fpFCs used for the identification of 300 individuals. The ROIs are listed based on how many robust edges were connected to the ROIs. The ROI labels were obtained from a 360 MMP parcellation map (Glasser et al., 2016), and the corresponding region from the AAL atlas with the highest overlap for each ROI is indicated (for more details, refer to the “ROIs on the AAL Atlas” section of the Supplementary Materials).

	FN	Hemisphere	ROI	AAL region	Count of connected edges
1	DMN	R	Area 9 anterior	Frontal_Sup_2_R	31
2	FPN	R	Area anterior 10p	Frontal_Sup_2_R	23
3	DMN	L	Area 9 anterior	Frontal_Sup_2_L	22
4	DMN	R	Area 9 Posterior	Frontal_Sup_2_R	21
5	FPN	R	Area 8C	Frontal_Mid_2_R	21
6	FPN	R	Area posterior 47r	Frontal_Mid_2_R	21
7	FPN	R	Area anterior 9‐46v	Frontal_Mid_2_R	20
8	FPN	L	Area 8C	Frontal_Mid_2_L	20
9	DMN	L	Area anterior 10p	Frontal_Sup_2_L	19
10	FPN	L	Area posterior 47r	Frontal_Inf_Tri_L	19
11	FPN	R	Area anterior 47r	Frontal_Mid_2_R	18
12	DAN	L	Area Lateral IntraParietal ventral	Parietal_Sup_L	17
13	DMN	L	Area 9 Posterior	Frontal_Sup_2_L	17
14	DMN	R	Area 10d	Frontal_Sup_Medial_R	16
15	DMN	L	Area anterior 47r	Frontal_Mid_2_L	16
16	DMN	L	Area posterior 10p	Frontal_Sup_2_L	16
17	FPN	R	Area IFJa	Frontal_Inf_Tri_R	15
18	FPN	R	Area 11l	OFCant_R	15
19	FPN	L	Area IFJa	Frontal_Inf_Tri_L	15
20	DAN	R	Area Lateral IntraParietal ventral	Parietal_Sup_R/Parietal_Inf_R	14
21	FPN	R	Area 8Av	Frontal_Mid_2_R	14
22	DAN	L	Premotor Eye Field	Precentral_L	14
23	DMN	L	Area 44	Frontal_Inf_Tri_L	14
24	DAN	R	Premotor Eye Field	Precentral_R	13
25	FPN	R	Area posterior 9‐46v	Frontal_Mid_2_R	13
26	DMN	L	Area 45	Frontal_Inf_Tri_L	13
27	FPN	L	Area anterior 9‐46v	Frontal_Mid_2_L	12
28	DMN	L	Superior 6‐8 Transitional Area	Frontal_Sup_2_L	12
29	DMN	R	Area TE1 Middle	Temporal_Mid_R	11
30	DMN	L	Area 8Av	Frontal_Mid_2_L	11

Abbreviations: AAL, automated anatomical labeling; DAN, dorsal attention network; DMN, default‐mode network; FN, functional network; fpFC, fingerprint of functional connectivity; FPN, frontoparietal network; HCP, Human Connectome Project; L, left; MMP, multimodal parcellation; R, right; ROIs, regions‐of‐interest.

3.4. Important edges for individual identification vary depending on the window length

For all window lengths, the intra‐ and inter‐FN edges of the FPN and DMN had the highest number of important edges (Figure 3c). When normalized by the number of total edges, the intra‐FPN and intra‐DAN also exhibited a high proportion of important edges. Although we reported that approximately 50% of the important edges were commonly found across window lengths, there were some noticeable changes in the important edges depending on window length. For example, there were a greater number of important but not robust intra‐FN edges in the VN and DAN as the window length decreased (Figure S12). On the other hand, a higher number of inter‐FN edges, such as those for FPN‐DMN, were important as the window length increased. The intra‐DMN and intra‐FPN also exhibited a slight change in the number of important edges between window lengths. From 25 repetitions instead of 5 repetitions of DNN training, 96.9, 96.4, 95.2, 32.8, and 11.5% of the total edges were replicated as important edges for the 15‐s, 30‐s, 1‐min, 3‐min, and 5‐min windows. When we lowered the threshold for ICC from 0.6 to 0.5, 96.9, 96.4, 95.2, 85.4, and 89.4% were replicated. Figure 4 shows the average fpFC across five window lengths for each of five arbitrarily chosen individuals (Figure 4a) and the fpFC for each window length (Figure 4b). The combination of positive and negative values for the edges appeared to characterize distinct fpFCs between individuals.

FIGURE 4.

FpFCs from five arbitrarily chosen individuals. (a) FpFCs showing the most significant of robust edges. (b) FpFCs showing the most significant of important edges for each of the five window lengths. Robust edges were important edges commonly observed across the five window lengths, and important edges were fpFC edges with ICC > 0.6 and STD > 99.5th percentile (instead of the 98th percentile for better visualization in the circular and glass brain plots). fpFC, fingerprint of functional connectivity; ICC, intra‐class correlation coefficient; STD, standard deviation.

3.5. The fpFC is not bound by a target group of individuals in the DNN

The fpFC of an individual was always closer to their own fpFC than to the fpFCs of other individuals across the tested configurations (Figure S3b). Specifically, others' fpFCs exhibited very low correlations (almost zero), while the target's own fpFC displayed much higher correlation. A qualitative examination of the robust edges' similarity to their corresponding fpFC revealed a tendency for the fpFC patterns to exhibit greater resemblance to their own rather than to those of others, irrespective of the configuration (Figure S3c). Moreover, the important edges for 15‐s window tvFC obtained from the DNN model used to identify 300 individuals heavily overlapped with those from the DNN models used to identify 100, 600, and 870 individuals, with an overlap of 70, 81, and 82%, respectively.

3.6. Genetic information is present in the fpFC

The mean of the z‐transformed Pearson's correlation coefficients of the fpFC for paired individuals was 0.13, 0.08, 0.06, and 0.00, for the MZ, DZ, sibling, and others groups, respectively (Figure 5). One‐way ANOVA revealed a significant group difference (df = 44,849, F = 82.5, p < 10⁻¹⁰). The similarities of the fpFC within the others group were significantly lower than for any of the family groups (df = 44,811, T = 12.2 for MZ; df = 44,804, T = 5.1 for DZ; df = 44,827, T = 8.6 for sibling; Bonferroni corrected p < 10⁻³ from a post hoc two‐sample t test). The similarities within the MZ group were more significant than within the sibling group (df = 42, T = 5.3, Bonferroni corrected p < 10⁻³) but the significance was reduced when compared to the DZ group (df = 19, T = 3.4, Bonferroni‐corrected p < .05) due to a potential outlier in the DZ group. These findings suggest that genetic information is reflected by the phenotypical fpFC.

FIGURE 5.

Similarity of the fpFC between paired individuals within each of four groups selected based on their genetic relatedness. Pearson's correlation coefficients followed by Fisher's r‐to‐z transform were used as a similarity measure. One‐way ANOVA was conducted on the four groups followed by a post hoc two‐sample t test. The resulting p‐values were Bonferroni‐corrected. fpFC, fingerprint of functional connectivity; MZ, monozygotic twins; DZ, dizygotic twins.

3.7. The proposed approach can be generalized to an independent dataset

The average error rate for the identification of the 314 individuals using 15‐s window tvFC was 15.9% when all hidden layers were frozen and only the output layer was trained (Figure S13). The error rate obtained for the HCP data and our institute was 14.5 and 45.2%, respectively. For a 30‐s window, the average error rate declined to 4.5% (3.4% for HCP and 27.3% for our institute). This error rate remains higher in comparison to the scenario where the model was trained with random initial weights, which showed an error rate of 4.1% (3.1 and 25.5%) for a 15‐s window and 0.6% (0.2 and 8.7%) for a 30‐s window. However, these rates were significantly above the level of chance (1–1/314 = 99.7%; one‐sample t test, p < 10⁻⁸) for both a 15‐s window (df = 299, T = −106.5 and df = 13, T = −13.7), and a 30‐s window (df = 299, T = −209.6 and df = 13, T = −14.9). This demonstrates that, despite not being flawless, the features learned for the individual identification of HCP subjects could hold informative value when applied to an independent dataset.

When parameters of all layers were fine‐tuned, the average error rate was substantially reduced to 4.2% (3.3% for HCP data and 24.3% for our institute) for a 15‐s window, and 0.4% (0.1 and 4.9%) for a 30‐s window. In addition, false positives between the subjects in the HCP and at our institute (i.e., falsely identified as an HCP subject for the subject at our institute) were substantially decreased for the transfer learning model when all DNN parameters were fine‐tuned. Interestingly, training completion could potentially have been expedited during the fine‐tuning process. For a 15‐s window, the average error rate at the 200th epoch was the lowest as 3.8%. Error rates for subjects from our institute were marginally lower (df = 13, T = −1.9, p = .09) in comparison to those for HCP participants, which did not show a statistically significant decrease (df = 299, T = −0.9, p = .4) relative to the random initialization case (Figure 6). For a 30‐s window, the lowest error rate during fine‐tuning was noted at the 600th epoch, whereas for the randomly initialized case, this low was observed at the 500th epoch. The error rates for fine‐tuning involving both HCP subjects and those from our institute were substantially lower (df = 299, T = −2.5, p = .01 and df = 13, T = −2.9, p = .01) compared with those of random initialization.

FIGURE 6.

(a) Error rates and (b) confusion matrix at the 200th epoch from individual identification using transfer learning with fine‐tuning of all parameters in the deep neural network (DNN) trained for the identification of 870 individuals using 15‐s windows. The total number of individuals to be identified was 314, which included 300 from the Human Connectome Project (HCP) and 14 from our institute. The dotted black horizontal line indicates the average error rate. The color bar for the confusion matrix was set at 6% to better visualize misclassified samples.

4. DISCUSSION

4.1. Summary of the study

We reported that tvFC from time windows as short as 15 s could be used to identify individuals with a DNN. The error rate from the identification of 300 individuals using a two‐hidden‐layer DNN (2.9 and 0.2% for 15 and 30 s, respectively) was substantially lower than that for alternative approaches (91.4 and 83.2% using PM; 5.1 and 0.4% using LR). The two‐hidden‐layer DNN also had the lowest error rate when compared to alternative models for 870 target subjects (95.1, 9.9, and 6.7% for the PM, LR, and DNN, respectively; 15‐s window). Individual fpFC, which characterizes the tvFC of individuals, contained important edges that were reliable and heterogeneous across the individuals. Approximately 50% of the important edges were robust, i.e., commonly found across the five window lengths (15 s, 30 s, 1 min, 3 min, and 5 min). The two most representative FNs associated with these robust edges were the FPN and DMN, many of which are located in the frontal lobe. We also raised the possibility that genotypical characteristics are reflected in the phenotypical traits of tvFC using stratified groups based on their genetic relatedness, which has been similarly reported in a recent study (Demeter et al., 2020). Finally, we demonstrated the feasibility of generalizing a DNN model trained using HCP data to an independent dataset acquired from our institute.

4.2. The advantage of DNN in individual identification

Overall, the reported findings suggest that a multilayered DNN model is more suitable for individual identification using tvFC patterns than alternative shallow approaches. In particular, PM demonstrated weaker performance overall than model‐based classifiers, probably because PM treats all edges equally and thus is susceptible to noise. On the other hand, model‐based approaches can weigh important edges by learning the corresponding weighting factors as model parameters. This feature‐learning capability has been suggested as a key advantage of deep learning approaches over standard ML approaches (Abrol et al., 2021). Thus, the superior performance of our DNN model is likely to be due to the inherent feature‐learning ability through nonlinear hidden layers. In contrast, shallow models such as LR learn only decision boundaries using the given input features.

Using PM, a considerable gap in error rates existed between the 1‐ and 3‐min windows. This may be explained with reference to a previous study demonstrating that the fidelity of FC fingerprints measured by ICC does not reach saturation until the window length reaches a size of 144 s to 288 s (Figure 2a) (Van De Ville et al., 2021). Although the DNN and LR models both performed almost perfectly for the 30‐s and longer windows (error rate <0.5%), the DNN for the 15‐s window exhibited a substantially lower error rate than did the LR. This suggests that individual‐specific information of FC reflected in BOLD signals between 0.033 Hz (=1/30 s) and 0.1 Hz (due to band‐pass filtering) is usefully exploited by both the DNN and LR. However, for BOLD signals between 0.067 Hz (=1/15 s) and 0.1 Hz, the DNN was better able to harness the information. Therefore, it appears that there may be relatively straightforward and crucial individual‐specific information contained within BOLD signals between 0.033 and 0.067 Hz. In contrast, individual‐specific information in BOLD signals between 0.067 and 0.1 Hz may be more subtle and noisier, which may be handled better by the DNN.

4.3. Weight‐sparsity‐optimized DNNs for effective information extraction

The identification performance was possibly higher because of the enhanced feature selection in each layer in our DNN model due to weight sparsity optimization within a grid search framework (Jang et al., 2017; Kim et al., 2016). Our findings highlight the importance of fine‐grained FC feature selection for individual identification from noisy tvFC patterns, particularly for short time windows (Figure 2a). A detailed discussion of weight sparsity can be found in “Optimization of Weight Sparsity for Enhanced Performance of DNN” in Supplementary Materials.

4.4. Shared features of tvFCs across window lengths detected by DNNs

Upon evaluating the tvFC patterns with a 15‐s window, DNN models trained on longer windows demonstrated error rates much lower than the chance level. Moreover, almost half of the important edges in the fpFC were commonly found across all window lengths. These observations imply that DNN models trained for different window lengths share FC features to a substantial degree, despite potential variability due to window length difference (Figure 3c). Upon performing multiple iterations of DNN, retention rates of important edges for 3‐ and 5‐min windows were only 32.8 and 11.5%, respectively, whereas retention exceeded 95% for the shorter windows. Interestingly, the retention rates for the 3‐ and 5‐min windows increased to 85.4 and 89.4% when the ICC threshold was lowered from 0.6 to 0.5. This indicates that the reliability of DNNs for longer windows might be compromised by the limited size of the training samples. Conversely, the specificity of the window length became evident through the analysis of error rates in tests using the shortest 15 s‐window. Specifically, the DNN trained across the 30 s, 1 min, 3 min, and 5 min windows exhibited a higher error rate compared with training solely with the 15 s window. This suggests that the individual‐specific FC feature in the shorter window may not be recovered from the increased number of training samples with longer window lengths (4 + 8 + 28 + 56 = 96).

4.5. Edges characterizing individuals in the fpFC

We demonstrated that individual‐specific characteristics of the FC are primarily embedded in the FPN and DMN, largely in the frontal lobe (Figure 3; Table 2). In particular, DLPFC in the FPN is known as one of the most important regions for cognitive control, and is believed to play a critical role in shaping individual traits in brain function (Goldberg, 2001; Miller & Cohen, 2001). Furthermore, DLPFC has also been found to be highly variable in its activation level, volume size, location, structural connectivity, and FC (Friedman & Miyake, 2017; Panikratova et al., 2020). For example, the FC of DLPFC has been shown to be related to individual variability in cognition (Panikratova et al., 2020; Turnbull et al., 2019). In line with these studies, we also found that the intra‐ and inter‐FN edges from the FPN and DMN were critical for robust identification of individuals across all window lengths (Figure 3c). Many previous studies have investigated representative FCs associated with individual traits. Finn et al. reported that regions in the FPN are important in forming personal fingerprints (Finn et al., 2015). In addition, Pallarés et al. found that the frontal and cingulate systems were important for individual identification (Pallarés et al., 2018). The DMN is known to be closely involved in internal mentation (Andrews‐Hanna, 2012). In particular, regions within the DMN have been associated with the retrieval of past experiences and the contemplation of future events during periods of rest, as well as in the processes of self‐reflection and understanding others. Based on this report, personal experiences and thoughts may have contributed to the FC features of the DMN in characterizing individuals in our findings. Furthermore, the DMN has been suggested as a network depicting individual differences regarding mind‐wandering or daydreaming (Gorgolewski et al., 2014; Kucyi & Davis, 2014). The highly robust individual‐specific edges in the fpFC in the higher‐order cortical areas could also be attributed to epigenetic factors during prolonged developmental maturation (Mueller et al., 2013; Petanjek et al., 2011). The regions are also known to be under substantial genetic influence, which is consistent with our results (Figure 5) (Colclough et al., 2017). Despite the dominant edges from the FPN and DMN in the fpFC, there was also considerable involvement of other FNs. Intra‐FN edges from the VN, SMN, DAN, and LN were relatively important for shorter window lengths (Figure 3c and Figure S12). Interestingly, the low inter‐individual variability for these FNs in resting‐state FC has previously been reported (Mueller et al., 2013). Inter‐FN edges such as VN‐DAN, LN‐FPN, and LN‐DMN appeared to be influenced by window lengths. Previous works using HCP data suggested the importance of higher‐order cognitive systems and sensorimotor systems in FC fingerprinting (Liu et al., 2018; Van De Ville et al., 2021). Notably, Van De Ville et al. found that VN and SMN are informative for shorter windows, whereas DMN and FPN are most informative for longer windows. Our findings are also in line with another work reporting FPN and DMN as containing the most reliable FC edges with respect to test–retest reliability (Noble et al., 2019). Similarly, we also demonstrated the importance of intra‐VN and intra‐DAN for the shorter window and FPN‐DMN for the longer window.

Our findings using Glasser's 355 ROIs were replicated using Schaefer's 100 and 200 ROIs with many robust edges in intra‐DMN and DMN‐FPN (Figures S9a,b and S10a,b). However, some differences were present, such as the relatively increased significance of intra‐VN, SMN‐DAN, or VAN‐DMN, and decreased significance of intra‐DAN, intra‐FPN, or DAN‐VAN for Schaefer's atlas compared to Glasser's atlas (Tozzi et al., 2020). One reason might be a lower spatial resolution in FC edges due to a reduced number of ROIs. A coarse scale of ROIs might have blurred a strong interaction of specific regions by blending them with other regions into a single big ROI (Termenon et al., 2016). The parcellation algorithms could lead to different boundaries of ROIs depicting different neural properties (Glasser et al., 2016; Schaefer et al., 2018). For example, the division of Glasser's ROIs is fine‐grained, while Schaefer's ROIs have a coarse division at the DLPFC of FPN (Figures S1a, S2a, and S2c). Despite overall finer ROI divisions for Glasser's ROIs, the early visual cortex at the occipital lobe is very coarsely divided in that scheme, whereas Schaefer's ROIs feature more fine‐grained divisions (Figures S1a, S2a, and S2c). Furthermore, a mismatch of network boundaries between the parcellation atlases could affect the interpretation of ROIs at the network level (Figures S1c, S2b, and S2d).

We confirmed that the fpFC obtained from the multiplication of weight feature vectors across the layers performed similarly well as the fpFC from activation maximization, one of the most popular methods for interpreting trained neural networks. This was indicated by the good agreement between the fpFCs obtained by either method (Figure S11a,b). Despite substantial overlap of robust edges, differences in fpFC existed between the two methods (Figure S11c). The fpFC edges between FPN‐VAN were found using the activation maximization approach but not by the multiplication of weight matrices. Compared to the activation maximization approach, the weight feature approach may have been lacking the nonlinear information learned by the DNN. It can be concluded that the fpFC from activation maximization would supplement the findings of fpFC from weight feature. However, the weight feature‐based approach has the benefit of instant availability once the DNN is trained, without requiring an additional iterative learning procedure. As an alternative to weight feature or activation maximization, explainability tools can be applied to ascertain the fpFC for each individual (i.e., output class) and identify important edges thereafter. For example, tools such as SHAP (Lundberg & Lee, 2017) or DeepLIFT (Shrikumar et al., 2017) can be used to evaluate the influence of each feature on the model output.

4.6. Influence of the surface registration scheme

Because FC was calculated using the averaged BOLD time series within each ROI, both the degree of alignment of the subject's EPI to a template and the ROI parcellation strategy may have influenced the results. We attempted to minimize potential misalignment by adopting the MMP scheme of MSMAll, which utilizes myelin maps from T1‐weighted and T2‐weighted images and rfMRI data to match a subject's native space to a group template on the surface. MSMSulc surface registration utilizes only cortical folding patterns (i.e., the sulcal depth) to match the subject's space to the group template. Thus, MSMAll is superior to MSMSulc for inter‐subject registration (Bijsterbosch et al., 2018; Dubois et al., 2018; Glasser et al., 2016). The tvFC patterns derived using MSMAll are expected to exhibit a relatively higher degree of authentic and subtle individual‐specific FC originating from neural activity by substantially reducing spurious yet relatively obvious individual‐specific FC patterns associated with individual‐specific anatomical cortical folding patterns (Bijsterbosch et al., 2018). This may explain why MSMAll registration resulted in a higher error rate than did MSMSulc (Figure S5). Fingerprints of cortical folding patterns during infancy have been reported for use in individual identification (Duan et al., 2020). Multimodal fingerprints utilizing both anatomical information from cortical folding patterns and functional information from tvFC patterns may thus further enhance individual identification performance.

4.7. Influence of confounding artifacts in the BOLD time series

We attributed the improvement using ICA‐FIX‐based time series denoising to the elimination of noise that obscured authentic individual‐specific neural activity (Griffanti et al., 2014; Salimi‐Khorshidi et al., 2014; Xifra‐Porxas et al., 2021). It is likely that cardiac and respiratory‐related components had already been mostly removed using ICA‐FIX denoising and/or band‐pass filtering (0.01–0.1 Hz) (Murphy et al., 2013) because of additional physiological noise correction via RETROICOR (Glover et al., 2000) did not alter the performance. It may be worth employing alternative physiological noise correction techniques such as ANATICOR (Jo et al., 2010). The fact that individuals with high head motions showed inferior performance than those with low head motions suggests that individual identification using the DNN was not based on the head motion‐related artifacts in the BOLD signals.

Using our DNNs, the test performance was consistently better than the validation performance (Figure S7). To interpret this finding, we conducted PM to evaluate the possibility that the tvFC patterns from the training data (R‐L and L‐R phase‐encoded rfMRI data on Day 1) were more similar to the test data (L‐R phase‐encoded rfMRI data on Day 2) than to the validation data (R‐L phase‐encoded rfMRI data on Day 2). PM results indicate that the error rates for the test data (91.4, 83.2, 65.8, 31.2, and 22.2% for windows of 15 s, 30 s, 1 min, 3 min, and 5 min, respectively) were lower than the error rates for the validation data (91.9, 84.2, 68.8, 40.8, and 29.5%). On Day 2, two rfMRI runs with the L‐R and R‐L phase‐encoding had been consecutively acquired in order (Smith et al., 2013). Since each run lasted about 15 min, there was a reasonable chance that participants might have been drowsy during the later run (i.e., the R‐L phase‐encoding) (Tagliazucchi & Laufs, 2014). Thus, the lower degree of vigilance and the higher likelihood of falling asleep might have degraded the quality of tvFCs in the validation set (Allen et al., 2014; Tagliazucchi & Laufs, 2014). Alternatively, the effect might be due to the varying signal quality depending on the phase‐encoding directions. Previous research suggests that the difference between R‐L or L‐R direction of phase‐encoding induces asymmetric dropout in the fMRI signal (Omidvarnia et al., 2022; Smith et al., 2013). Specifically, the R‐L phase‐encoding direction is known to have a more severe dropout in the left hemisphere than the L‐R phase‐encoding direction, which might lead to increased distortion of the FC edges in the left hemisphere under that encoding (Mori et al., 2018). This may therefore constitute another possible explanation of the slightly increased error rates from the test data compared to the validation data.

4.8. Generalization to an independent dataset

Fine‐tuning the classification of varying numbers of target subjects can be achieved through the utilization of our pretrained DNN from the input layer to the last hidden layer, coupled with the incorporation of a new output layer. This method is effective as the fpFC for each individual can be uniquely identified through feature extraction using our DNN. More specifically, as previously noted, the fpFCs of a single subject demonstrated considerable consistency across DNN models trained on different quantities of target subjects (i.e., 300 and 600) (Figure S3). Additionally, our findings demonstrate the capacity of our pretrained DNN to employ transfer learning for predicting a larger population of target subjects (Figures 6 and S13). Given the identification of important and robust FC edges for individual classification, we postulate that our DNN model, pretrained with the HCP dataset, can be effectively adapted for classifying alternative sets of subjects by fine‐tuning the pretrained DNN. Nonetheless, the number of subjects substantially exceeding the quantity of hidden nodes (i.e., 2000 in our study) may lead to challenges due to the possibility of underrepresented feature representations when the hidden nodes are outnumbered by output nodes. To mitigate this, expanding the network's capacity while maintaining a constant count of input nodes by adopting techniques that ensure over‐complete representation at the hidden nodes (Jiang et al., 2019; Le et al., 2011; Vincent et al., 2010) could be a promising strategy.

We generalized our DNN model trained using the HCP rfMRI dataset to a rfMRI dataset acquired from our institute. Despite the significantly higher error rates for the data collected from our institute compared with those for the HCP data (Figures 6 and S13), the resulting performance was significantly better than would be expected by chance. Compared to the scenario of freezing hidden layers, fine‐tuning all layers of the DNN enabled the fine‐grained adjustment of model parameters for the samples in the new dataset. Consequently, fine‐tuning substantially reduced error rates for both datasets. Moreover, fine‐tuning resulted in much faster convergence but marginally lower error rates for the 15‐s window and similar convergence speed but substantially lower error rates for the 30‐s window, when compared to the DNN trained with randomly initialized parameters. This demonstrated the advantage of transfer learning in the prediction of an unseen dataset by increasing the generalization performance of the DNN.

The lower performance compared to the HCP data may not be due to the limited capability of our DNN model but rather to the lower quality of our rfMRI measurements, which were taken using inferior hardware specifications. We used a 3 T Trio MRI scanner for our institute data, while the HCP data is from a Connectome Skyra scanner. We also used a 12‐channel head coil, while a 32‐channel head coil was used for the HCP data. Our acceleration factor for parallel imaging was 4 (i.e., MB factor of 2 and GRAPPA) compared with 8 for the HCP (MB factor = 8, GRAPPA off). Due to this lower acceleration factor, we increased the TR to 1.44 s for our data (compared to 0.72 s for the HCP) and increased the voxel size to 3 mm isotropic (compared to 2 mm isotropic for the HCP).

4.9. Potential weaknesses and further works

Our study does not include a comparison with recent works on individual identification via FC patterns, which presents a challenge in comparing performance metrics without bias due to variables such as dataset disparities, subject quantity, preprocessing procedures, and variations in temporal window lengths. Previous studies have demonstrated a correlation between individual‐specific features in static FC and fluid intelligence levels (Finn et al., 2015), and an association between features in dynamic FCs and cognitive task performance (Fong et al., 2019). Consequently, a promising avenue for future research is to explore the potential connections between the fpFC identified in our investigation and cognitive or behavioral attributes. Our findings, which indicate a greater similarity among genetically close individuals and inferior classification performance in classifying individuals from the HCP dataset to those from our institution using transfer learning, suggest that our DNN model could facilitate meaningful classifications. For example, a DNN trained to classify a specific clinical cohort might be deployed to determine whether new individuals correlate with that clinical group or control samples, based on the prediction performance of the transfer learning scheme. Alternatively, clustering of the fpFC patterns or hidden representations could reveal potential sub‐groups within the training samples, using the DNN designed for individual classification.

We only utilized BOLD signals from the cerebral cortical areas. The subcortical regions and cerebellum have also been reported to play a range of critical functional roles, from those essential for living to those associated with high‐level cognitive processes, such as coordinating thought (Koziol et al., 2014). Future research that can systematically evaluate the importance of FNs across the cerebral cortex, subcortex, and cerebellum is thus warranted (Van De Ville et al., 2021). The FC of rfMRI data measures simultaneous co‐activation across ROIs using BOLD signals that measure neural activity indirectly via neurovascular coupling between brain tissue and blood vessels (Huettel et al., 2009). Therefore, it is important to determine whether the main contributor to individual identification using tvFC patterns is distinct and individual‐specific characteristics associated with spontaneous neural activity during a resting state or those associated with the hemodynamic response function representing neurovascular coupling (Buxton et al., 2004; Gonzalez‐Castillo et al., 2014; Handwerker et al., 2012).

Because we used nonoverlapping windows in calculating tvFC patterns from BOLD signals, the number of training samples varied by window length since the scan duration was fixed. Despite the expectation that each FC pattern would be less noisy and clearer under a given longer versus a shorter window length, we observed a higher error rate for a 5‐min window (0.2%) than for a 3‐min window (0.1%). This may potentially be due to overfitting caused by the smaller number of training samples for the 5‐min window than for the 3‐min window. Thus, an interesting and important question is whether there is an optimal window length for individual identification using DNN if the number of samples is identical across different window lengths. Our motivation for using nonoverlapping rectangular windows was to reduce potential dependency between adjacent tvFC patterns caused by partially overlapping/shared BOLD signals between consecutive temporal windows. Alternatively, it would also be possible to adopt a standard approach of sliding a window with a temporal overlap between the BOLD signal segments since the training, validation, and test samples are separated by the fMRI sessions. This will increase the number of training samples that may work as data augmentation schemes for the DNN training, which may alleviate a potential overfitting issue. Additionally, alternative window shapes such as Tapered Gaussian or Hanning windows can be utilized to further reduce potential discontinuity and/or abrupt changes between tvFC patterns (Preti et al., 2017).

We endeavored to validate the DNN's identification performance and to ascertain reliable edges by iterative training across numerous cycles. It is worth noting, however, that there could still be the risk of underspecification. This suggests that our model may not have found an overarching, generalized solution applicable across various domains, or there may exist alternative solutions capable of addressing the problem (D'Amour et al., 2022). In this context, future research aimed at mitigating the risk of underspecification risk is warranted. There are important alternative options beyond our proposed method of weight sparsity optimization to prevent overfitting DNNs and to improve their performance. For example, transfer learning has become feasible owing to the availability of extensive open fMRI datasets such as ABCD (Casey et al., 2018), UK Biobank (Sudlow et al., 2015), and Healthy Brain Network Biobank (Alexander et al., 2017). This approach may involve the direct transfer of task‐related knowledge (in our case, individual identity) through supervised pretraining‐based transfer learning, or the acquisition and transfer of knowledge unrelated to the downstream task from the source data via self‐supervised pretraining‐based transfer learning. Studies have shown that implementing transfer learning in deep learning models using fMRI data has enhanced model performance on downstream tasks (Hwang et al., 2023; Li et al., 2018; Rahman et al., 2022).

Each output unit in our DNN represents a distinct individual in our training dataset; therefore, the model cannot incorporate individuals outside of this dataset. Specifically, the FC pattern of an unrecognized individual will be classified as the most similar existing individual in the training set based on the closest matching fpFC. This approach is suboptimal for individual classification across a broader population when the trained DNN is applied. To address this shortcoming, integrating an out‐of‐distribution (OOD) detection module into the DNN has been proposed, which is in line with recent discussions estimating uncertainty within deep learning models (Hendrycks & Gimpel, 2016; Yang et al., 2021). Enhancing our DNN with an OOD detection module that includes a confidence score could substantially improve the model's reliability for individual identification. Finally, our DNN model was a fully connected NN and thus received 1D vectorized off‐diagonal elements of the FC matrix as input. Our model architecture may hence be limited in accommodating the 3D structure of the brain. Future studies using a deep learning model more suitable for dealing with FC data are warranted. For example, a graph neural network incorporating the brain's topological architecture or LSTM/RNN to account for temporal changes could be used to evaluate whether the prediction performance and identified fpFCs can be further optimized (Farahani et al., 2019; Kawahara et al., 2017).

5. CONCLUSIONS

Our study offers an insight into the discovery of the intrinsic mode of the human brain using whole‐brain resting‐state FC and weight‐sparsity‐controlled DNNs. This study evaluated the identification performance of 100–870 target subjects (depending on the scenario), while mainly focusing on 300 subjects. We demonstrated that a DNN can enhance accuracy by systematically comparing its performance with alternative approaches across different scenarios of window lengths and numbers of target subjects to identify. Moreover, we evaluated the reliability and robustness of the fpFC across the varying configurations and interpreted our findings in the context of functional connectome. We also raised the possibility that genotypical characteristics are reflected in the phenotypical traits of tvFC using stratified groups based on their genetic relatedness. In addition, the feasibility of generalizing a DNN model trained using HCP data to an independent dataset acquired from our institute was demonstrated in the context of transfer learning. Finally, our study is valuable in that it extends the investigation of FC fingerprinting to several hundreds of subjects (870 in this study) compared to the limited number of subjects in the previous works (Finn et al., 2015; Liu et al., 2018; Van De Ville et al., 2021). The code and sample results of fpFC are available at https://github.com/bsplku/fingerprintFCdnn.

AUTHOR CONTRIBUTIONS

Juhyeon Lee and Jong‐Hwan Lee conceptualized the design of the experiments and methodology. Juhyeon Lee analyzed the data. Juhyeon Lee and Jong‐Hwan Lee contributed to the writing of the manuscript.

FUNDING INFORMATION

This work was supported by the National Research Foundation (NRF) grant, Ministry of Science and ICT (MSIT) of Korea (NRF‐2017R1E1A1A01077288, 2021M3E5D2A0102249311, No. RS‐2023‐00218987), and in part by the Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean Government (23ZS1100, Core Technology Research for Self‐Improving Integrated Artificial Intelligence System).

CONFLICT OF INTEREST STATEMENT

The authors have no conflicts of interest regarding this study, including financial, consultant, institutional, or other relationships. The sponsors had no involvement in the study design, data collection, analysis or interpretation of the data, manuscript preparation, or the decision to submit for publication.

Supporting information

DATA S1 Supporting Information.

Lee, J. , & Lee, J.‐H. (2024). Discovering individual fingerprints in resting‐state functional connectivity using deep neural networks. Human Brain Mapping, 45(1), e26561. 10.1002/hbm.26561

DATA AVAILABILITY STATEMENT

All Human Connectome Project (HCP) data used in the present study are available for download from the HCP website (www.humanconnectome.org). The rfMRI data acquired at Korea University are available from the corresponding author upon reasonable request.