Analysis of Longitudinal Change Patterns in Developing Brain Using Functional and Structural Magnetic Resonance Imaging via Multimodal Fusion

ABSTRACT

Functional and structural magnetic resonance imaging (fMRI and sMRI) are complementary approaches that can be used to study longitudinal brain changes in adolescents. Each individual modality offers distinct insights into the brain. However each individual modality may overlook crucial aspects of brain analysis. By combining them, we can uncover hidden brain connections and gain a more comprehensive understanding. In previous work, we identified multivariate patterns of change in whole‐brain function during adolescence. In this work, we focus on linking functional change patterns (FCPs) to brain structure. We introduced two approaches and applied them to data from the adolescent brain and cognitive development (ABCD) dataset. First, we evaluate voxel‐wise sMRI‐${\mathit{FCP}}_{\mathit{\text{asym}}}$ coupling to identify structural patterns linked to our previously identified FCPs. Our approach revealed multiple interesting patterns in functional network connectivity (FNC) and gray matter volume (GMV) data that were linked to subject‐level variation. ${\mathit{FCP}}_{\mathit{\text{asym}}}$ components 2 and 4 exhibit extensive associations between their loadings and voxel‐wise GMV data. Secondly, we leveraged a symmetric multimodal fusion technique called multiset canonical correlation analysis (mCCA) + joint independent component analysis (jICA). Using this approach, we identified structured ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ such as one showing increased connectivity between visual and sensorimotor domains and decreased connectivity between sensorimotor and cognitive control domains, linked to structural change patterns (${\mathit{\text{SCPs}}}_{\mathit{sym}}$) including alterations in the bilateral sensorimotor cortex. Interestingly, females show stronger connection between brain functional and structural changes than males, highlighting gender‐related differences. The combined results from both asymmetric and symmetric multimodal fusion methods underscore the intricate gender‐specific nuances in neural dynamics. By utilizing two complementary multimodal approaches, our study enhances our understanding of the evolving nature of whole brain connectivity and structure during the adolescent period, shedding light on the nuanced processes underlying adolescent brain development.

In this study, we propose both asymmetric and symmetric multimodal fusion techniques to investigate longitudinal brain change patterns in adolescents. In our analysis, multivariate patterns of functional network connectivity and gray matter volume (GVM) demonstrate relationships with increased age. In asymmetric multimodal fusion, two functional change patterns (FCP) components exhibit extensive associations between their loadings and voxel‐wise GMV data. In symmetric multimodal fusion, one FCP component shows increased connectivity between visual and sensorimotor domains and decreased connectivity between sensorimotor and cognitive control domains, linked to structural change patterns including alterations in the bilateral sensorimotor cortex.

1. Introduction

The intricate neural network within the brain is recognized as one of the most complex systems in existence. A modern neuroscience approach involves viewing brain regional interactions as a network, known as the brain connectome or brain network (Kong and Yu 2014). By understanding network properties, researchers can gain insight into how the brain manages information flow by transferring neural signals among brain regions (Mesulam 2000). Magnetic resonance imaging (MRI) is a tremendously used method for obtaining comprehensive brain information and one of the only modalities that is capable of visualizing both structure and function. Structural neuroimaging modalities, such as sMRI and diffusion MRI, offer insights into the brain's anatomical structure and tissue composition, while functional neuroimaging modalities, such as fMRI based on the blood‐oxygenation‐level‐dependent signal, provide indirect measurements of brain function and activity (Abrol et al. 2017; Ogawa et al. 1990). fMRI is a key method for assessing functional connectivity (FC) by analyzing temporal coherence among different brain regions using resting‐state data. Most studies have still examined functional and structural measures separately when analyzing the brain. However, there is considerable evidence that combining structural and functional MRI data can lead to benefits (Calhoun and Adali 2008; Rykhlevskaia et al. 2008). Multimodal fusion in neuroimaging integrates information obtained from various imaging techniques, aiming to overcome individual modality limitations and gain deeper insights into brain structure and function (Calhoun and Sui 2016; Sui et al. 2014; Zhang et al. 2011). The primary goal of multimodal fusion is to enhance the analytical capabilities of each modality through combined analysis, rather than treating each modality separately.

Prior research in multimodal analysis often involves examining data from distinct modalities separately and later combining the independent outcomes obtained from these individual analyses. Alternatively, some studies have used information from one modality to constrain or guide models related to another modality. While these approaches have been beneficial, they can underutilize cross‐modality information. There is a growing trend in approaches that employ symmetric data fusion methods (Calhoun and Sui 2016) to capitalize on joint information across modalities. Feature‐based symmetric data fusion methods initially extract valuable, high‐dimensional features from diverse modalities and then explore the connections among these features. This method effectively uses the complementary information present in each modality to reveal variations in data that might not be evident through unimodal analyses. Various studies have highlighted the potential of employing such cross‐modality or joint information in understanding the human brain and its disorders. These methods have been instrumental in characterizing diseases, identifying potential biomarkers, and unraveling disrupted connections in complex mental illnesses (Calhoun and Sui 2016).

Each multivariate fusion method possesses distinct optimization priorities and limitations. For instance, methods like mCCA (Correa et al. 2009) and partial least squares (PLS) (Chen et al. 2009; Lin et al. 2003) facilitate both common and distinct levels of connection across modalities. However, these methods may not achieve adequate spatial sparsity in their separated sources. For instance, mCCA emphasizes intersubject covariation across two feature sets, generating linked variables known as canonical variants (CVs). These CVs correlate with each other solely on the same indices, and their corresponding correlation values are termed canonical correlation coefficients (CCC). While this approach captures both common and distinct aspects of features, the resulting brain maps for multiple components may appear similar if the CCCs lack sufficient distinctiveness. On the other hand, spatial decomposition approaches like jICA (Calhoun et al. 2006) and linked independent component analysis (Groves et al. 2011) maximize independence among estimated sources combining multiple modalities, but they primarily allow a common mixing matrix. While these methods detect common features across all modalities effectively, they may neglect features distinct to one or more modalities, especially when combining more than two modalities. Several prior studies that merged function and structure (Camara et al. 2010; Olesen et al. 2003; Rykhlevskaia et al. 2008; Sui et al. 2012) support the idea that components derived from each modality exhibit some correlation in their mixing profiles among subjects. This serves as motivation to utilize an approach that aims for optimal intermodal association flexibility while ensuring robust source separation capabilities.

To analyze shared information among features from different imaging modalities, we employed the multiset canonical correlation analysis + joint independent component analysis (mCCA + jICA) method (Sui et al. 2011). This widely recognized and extensively used data‐driven multivariate multimodal fusion approach (Calhoun et al. 2006; Correa et al. 2010) integrates mCCA and jICA in a two‐step process (Calhoun and Sui 2016). Initially, mCCA identifies highly correlated components across multiple modalities (Sui et al. 2011; Sui et al. 2013). Subsequently, jICA decomposes these correlated components into spatially independent components (ICs). The mCCA + jICA algorithm has been utilized (Sui et al. 2011) to combine fMRI contrast maps and diffusion tensor imaging (DTI) fractional anisotropy (FA) maps, enhancing the accuracy of group classification among healthy controls (HCs), schizophrenia patients (SPs), and bipolar patients (BPs) compared to using the algorithms individually. Previous applications include Ouyang et al., who identified patterns of gray matter (GM) and white matter (WM) covariance in Alzheimer's disease patients (Ouyang et al. 2015). Similarly, Kim et al. applied mCCA + jICA to analyze multimodal sMRI and DTI data from HCs and patients with obsessive–compulsive disorder, uncovering significant alterations in interconnected networks of GM and WM (Kim et al. 2015). But, to the best of our knowledge, no prior studies have investigated the estimation of changes in multivariate pattern coupling in FNC and GMV associated with age progression, utilizing the mCCA + jICA multimodal fusion analysis method.

In our study, we present two innovative methodologies: (1) explore the relationship between multivariate functional change patterns (${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$) and voxel‐wise gray matter volume ($\mathrm{\Delta }$GMV) data to investigate age‐related changes in whole‐brain structure and function within individuals (2) employ a symmetric multimodal fusion technique mCCA + jICA to uncover structural change patterns (${\mathit{\text{SCPs}}}_{\mathit{sym}}$) associated with ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ using FNC matrices and GMV data from the ABCD dataset, comprising over 11,000 adolescent subjects across multiple scans. For each subject, we compute cell‐wise $\mathrm{\Delta }$FNC and $\mathrm{\Delta }$GMV matrices, followed by estimating covarying multivariate patterns. Without predefined restrictions to specific seed regions, we calculate voxel‐wise correlations between functional data loading parameters and $\mathrm{\Delta }$GMV data. The second approach involves using the mCCA + jICA multimodal fusion technique to estimate covarying multivariate patterns between ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ and ${\mathit{\text{SCPs}}}_{\mathit{sym}}$.

Through both symmetric and asymmetric multimodal fusion techniques, our analysis identifies FNC linked to GMV data, suggesting concurrent changes between functional connectivity and structural data during adolescence. Notably, gender‐related differences reveal stronger coupling between brain functional and structural changes in females compared to males. Our statistical analysis unveils several FCPs, and SCPs associated with longitudinal changes in psychopathology and cognition scores within the developing brain. The remainder of the research paper is organized as follows: in the Materials and Methods section, we present the data preprocessing and analysis procedures. In the Results section, we showcase changes in brain functional and structural coupling with age and the connections between various FCPs and SCPs with gender, differences in psychopathology, and cognition scores. Finally, in the Discussion and Conclusion section, we delve into the implications of our findings.

2. Materials and Methods

2.1. ABCD Data Summary

The study utilizes data collected by the ABCD study ¹ . The primary objective of the ABCD study is to monitor brain development during adolescence. To realize this, the study collected a diverse array of data to discern the influences of biological and environmental factors on developmental trajectories. The ABCD study is the largest long‐term investigation of brain development and child health in the United States. It includes multisession MRI scans from over 11,800 children aged 9–11 years at baseline. Table 1 provides the demographic information for the ABCD dataset. The dataset covers subject details such as social, emotional, and cognitive development, gender identity, physical and mental health assessments, and medical backgrounds. Ethical considerations were upheld through parental informed consent and child assent, all approved by the Institutional Review Board (IRB). The ABCD dataset is accessible via the National Institute of Mental Health Data Archive (NDA) ² , which has been made available as an open‐source resource following its compilation from a diverse range of research endeavors across various scientific domains. Collaborating with major MRI system manufacturers (Siemens, General Electric, and Philips), the data were collected from 21 sites across the United States, ensuring standardized imaging methods. The TR was 800 ms, with a resolution of 2.4 $\times$ 2.4 $\times$ 2.4 mm. More information about imaging parameters can be found at ³ . The quality of data was maintained through standard fMRI preprocessing and the NeuroMark framework (Du et al. 2020), a fully automated independent component analysis (ICA)‐based approach that identifies brain networks across subjects. The present study utilized data from 2734 subjects who have both baseline and 2‐year follow‐up scanned data of both the FNC and gray matter volume data. In our analysis, we specifically focused on the first scan from both the baseline and 2‐year follow‐up data.

TABLE 1.

ABCD data demographic information.

Event name	Subject	Gender (F/M)	Age (month)	Weight (lb)	Height (inch)	Race (W/H/B/O/A)
Baseline	11,244	5347/5833	119 ± 8	82.5 ± 23.7	55.3 ± 3.3	4771/1863/1325/929/181
Second year	3678	1589/1875	143 ± 8	106.6 ± 31.4	60.1 ± 3.8	1631/629/317/284/57

Abbreviations: A, Asian; b, black; F, female; H, Hispanic; M, male; O, others/unknown; W, white.

2.2. Image Preprocessing of FMRI Data

We conducted preprocessing on the FastTrack fMRI images using a combination of the FMRIB Software Library v6.0 (FSL) toolbox and Statistical Parametric Mapping 12 (SPM) toolbox within the MATLAB 2019b environment. Initially, we corrected for rigid body motion by employing the mcflirt tool in FSL. Then, we conducted distortion correction utilizing fMRI field map data that were collected with phase‐reversed blips. This process generated pairs of images with distortion occurring in opposing directions. To estimate the susceptibility‐induced off‐resonance field, we employed the FSL tool topup, utilizing volumes acquired with phase encoding both in the anterior–posterior and posterior–anterior directions. The coefficients derived from the output field map were then applied to correct distortion in the fMRI volume using the FSL tool applytopup. In the following step, we removed the initial 10 scans with substantial signal changes to allow the tissue to stabilize in terms of radio frequency excitation. Subsequently, we spatially aligned the fMRI data to the standard Montreal Neurological Institute (MNI) space, utilizing the echo‐planar imaging template and resampling the data to 3 $\times$ 3 $\times$ 3 mm isotropic voxels via the spatial normalization tool in SPM. Lastly, we applied Gaussian smoothing with a full width at half maximum (FWHM) of 6 mm to the resliced fMRI images.

2.3. Quality Control (QC)

We conducted data quality control (QC) on the preprocessed fMRI images to select subject data for subsequent analysis. The quality of how well subjects' data were normalized to the MNI space has a significant impact on both the results of ICA and the estimation of FNC. Consequently, we excluded scans that did not exhibit satisfactory normalization to the MNI standard space. To be more specific, we compared individual masks with a group mask and retained scans that demonstrated strong similarities between their individual masks and the group mask. To achieve this, we initially calculated an individual mask for each scan of each subject based on the first fMRI time volume. Voxels were set to 1 if they exceeded $90\%$ of the mean signal across the entire brain. Subsequently, we generated a group mask by designating voxels as 1 if they had more than $90\%$ agreement with individual masks across the scans. For each scan, we computed spatial correlations between the group mask and the individual mask, focusing on the top 10 slices, the bottom 10 slices, and the entire mask. This resulted in three correlation values for each scan. We included scans for further analysis if they met the following criteria: top‐10‐slices correlation greater than 0.75, bottom‐10‐slices correlation exceeding 0.55, and whole‐brain correlation surpassing 0.8. This method ensures the inclusion of high‐quality masks and fMRI data for retained scans, building on its success in previous research.

2.4. Neuromark Framework

To capture reliable intrinsic connectivity networks (ICNs) and their corresponding time courses (TCs) across each subject, we utilized a robust and fully automated independent component analysis (ICA)‐based framework known as NeuroMarkfMRI_—1.0 network templates was applied to the ABCD data. The NeuroMark_—fMRI_—1.0 network templates were constructed based on two healthy control datasets: the Human Connectome Project (HCP, 823 subjects after selection) and the Genomics Superstruct Project (GSP, 1005 subjects after selection). Further details about the NeuroMark framework and templates are available in the GIFT toolbox ⁴ and at ⁵ (Du et al. 2020). The selected spatial priors have been shown to be highly reliable across different pipelines and various adult and adolescent datasets and populations (DeRamus et al. 2021). This approach yields 53 ICNs for each subject, with the resulting networks being highly consistent and comparable across subjects, sessions, and scans. Children's data can be noisy with more confounding effects, such as larger head motions. To mitigate this, we included four additional postprocessing steps to carefully regress out the remaining noise in the time courses (TCs) of the ICNs: (1) detrending linear, quadratic, and cubic trends, (2) removing detected outliers, (3) performing multiple regression to remove variance linked to head motion parameters (three rotations and three translations) and their derivatives, and (4) applying bandpass filtering with a cutoff frequency of 0.01–0.15 Hz. After the postprocessing, we calculated Pearson correlation coefficients between postprocessed TCs to estimate the static FNC for each scan.

2.5. Preprocessing of SMRI Data

We conducted preprocessing on the sMRI data using statistical parametric mapping ⁶ within the MATLAB 2020b environment. The structural images were subjected to segmentation into gray matter, white matter, and CSF with additional modulation by the Jacobian to produce voxel‐wise gray matter volume (GMV) maps. Subsequently, the GMV maps underwent smoothing via a Gaussian kernel with a FWHM of 6 mm.

2.6. Analysis of Longitudinal Change Patterns in FNC and GMV

In our research, we utilized subject‐specific fMRI and sMRI data acquired during both the baseline and 2‐year follow‐up scans to investigate changes in FNC and GMV. We calculated the cell‐wise differences between the baseline and 2‐year follow‐up FNC and GMV data to create $\mathrm{\Delta }$FNC and $\mathrm{\Delta }$GMV matrices, respectively, representing the changes in FNC and GMV over time. These matrices were then analyzed using both asymmetric and symmetric fusion approaches to find link between brain functional connectivity and structure. In the asymmetric fusion approach, we applied the ICA using the infomax algorithm (Bell and Sejnowski 1995) to deconstruct the $\mathrm{\Delta }$FNC in order to recognize longitudinal brain functional coupling and capture covarying patterns of changes, which are called ${\mathit{FCP}}_{\mathit{\text{asym}}}$. We extended our analysis to investigate the connection between ${\mathit{FCP}}_{\mathit{\text{asym}}}$ and brain structure. This involved calculating the voxel‐wise correlation between the raw $\mathrm{\Delta }$GMV data and the loading parameters obtained from functional data after ICA estimation. More precisely, the second‐level ICA model equation can be expressed as:

$$X=A\cdot S$$ (1)

The following effectively represents the functional input data for the ICA model as:

Next, we applied a symmetric fusion approach via mCCA + jICA to estimate joint ${\mathit{\text{SCPs}}}_{\mathit{sym}}$ and ${\mathit{\text{FCPs}}}_{\mathit{sym}}$. In the symmetric fusion approach, we run mCCA + jICA method to deconstruct the $\mathrm{\Delta }$FNC and $\mathrm{\Delta }$GMV matrices and identify patterns of change, namely, functional and structural change patterns for FNC and GMV, respectively. We determined the optimal number of components, selecting five components for both GMV and FNC data using the elbow criteria. The mCCA + jICA model equation utilized in our experiment is expressed as follows:

$$X_k=A_k\cdot S_k$$ (2)

In this analysis, the data matrix X has dimensions of 2734 (subjects) $\times$ cells (representing either the upper triangular elements of the $\mathrm{\Delta }$FNC matrix for fMRI or the number of voxels for sMRI). The matrix A has dimensions of 2734 $\times$ 5 (components), S is 5 $\times$ cells (components), and the mCCA + jICA model involves k = 2 modalities.

This effectively represents the input data for the mCCA + jICA approach as:

Here, $\mathrm{\Delta }$FNC denotes the difference between baseline ($F_0$) and 2‐year follow‐up ($F_2$) functional network connectivity data, while $\mathrm{\Delta }$GMV corresponds to the difference between baseline ($G_0$) and 2‐year follow‐up ($G_2$) gray matter data. The source matrices of FCPs and SCPs capture maximally independent patterns of functional and structural changes, respectively. The parameter $a_i$ is each component's subject‐specific loading parameters of FNC data in the ICA model. The terms $a_F$ and $a_G$ denote the subject‐specific loading parameters for each component in the FNC and GMV data, respectively, in the context of the mCCA + jICA approach. These loading parameters quantify the individual subject's contribution to the respective components. A block diagram of the analysis workflow is shown in Figure 1. Moreover, following the mCCA + jICA estimation, we proceeded to evaluate the loading parameters and source matrix. In order to identify ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ and ${\mathit{\text{SCPs}}}_{\mathit{sym}}$ that showed significant longitudinal changes compared to zero, we conducted one‐sample t‐tests on the loading parameters $a_F$ and $a_G$ for both modalities. We also conducted a one‐sample t‐test on the loading parameters ($a_i$) following the ICA estimation. Statistical significance was assessed at a $95\%$ confidence level, with adjustments made for multiple comparisons using a false discovery rate (FDR) approach.

FIGURE 1.

Block diagram of longitudinal multimodal functional and structural change pattern coupling analysis workflow. The subject‐specific $\mathrm{\Delta }$FNC and $\mathrm{\Delta }$GMV matrices were constructed by calculating the difference between preprocessed baseline and 2‐year FNC and GMV data.

2.7. Gender‐Based Multimodal Fusion Analysis

For the asymmetric multimodal fusion approach, we calculated correlations between $\mathrm{\Delta }$GMV data and loading parameters of functional data separately for males and females to examine gender differences in coupling. The next step involved computing the difference between these correlations to understand the gender effect. In the symmetric multimodal fusion approach, we segregated the male and female loadings for both GMV and FNC data. Subsequently, we computed correlations between GMV male loadings and FNC male loadings, and between GMV female loadings and FNC female loadings. By quantifying the difference between these correlations (female—male), we assessed the strength of coupling between GMV and FNC loadings relative to gender. In both approaches, a significant positive difference would indicate a stronger coupling between functional and structural data in females compared to males.

2.8. Quartile Analysis of Pattern Changes via Relationship to Subject Measures

In order to reduce the number of comparisons, we calculated composite scores for both cognitive and psychopathology assessments. We used the same variables and procedures to compute the composite cognitive and psychopathology assessments as in our previous work (See supplementary material for details) (Fu et al. n.d.; Saha et al. 2024). Specifically, we computed the subject‐wise differences between baseline and 2‐year data for composite psychopathology and cognitive scores, representing changes in psychopathology and cognitive performance, respectively. To explore the relationship among subjects who displayed the most significant age‐related change pattern association with structural data, we conducted an analysis of the connection between loading parameters of ICA and $\mathrm{\Delta }$GMV data. We specifically selected subjects whose changes in psychopathology scores fell within or below the lower quartile. We then computed the correlation between the loading parameters of ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ and the raw $\mathrm{\Delta }$GMV data for these chosen subjects, creating a voxel‐wise association map for the lower quartile. Following the same procedure for subjects in the upper quartile, we also generated an upper quartile voxel‐wise association map. Finally, we calculated the disparity between the upper and lower quartile voxel‐wise correlation maps. The same methodology was applied to evaluate differences in cognitive scores. It is important to note that all our findings underwent correction for multiple comparisons using the FDR (Benjamini and Hochberg 1995).

3. Results

3.1. Structure–Function Coupling Strengthens With Age Across Development

The NeuroMark_—fMRI_—1.0 template encompasses a total of 53 reproducible networks, categorized into seven domains based on their anatomical and functional attributes. These domains include subcortical, auditory, sensorimotor, visual, cognitive control, default mode, and cerebellar domains (Du et al. 2020). The experimental outcomes from the asymmetric fusion approach, involving spatial maps illustrating the links between multivariate ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ and voxel‐wise GMV data, are presented in Figure 2. In this figure, five ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ components are plotted along with the spatial maps showcasing the voxel‐wise correlations between GMV data and ${\mathit{FCP}}_{\mathit{\text{asym}}}$ loadings. Our symmetric fusion approach results, depicted in Figure 3, exhibit spatial maps illustrating the connections between multivariate ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ and ${\mathit{\text{SCPs}}}_{\mathit{sym}}$. This figure showcases five ${\mathit{FCP}}_{\mathit{sym}}$ components and their corresponding spatial maps of ${\mathit{SCP}}_{\mathit{sym}}$ components. Furthermore, the associations of ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ and ${\mathit{\text{SCPs}}}_{\mathit{sym}}$ components with age are depicted using upper and lower arrows along with their associated T‐values. A high negative (or positive) T‐value signifies an increase (or decrease) in the expression of the specific change pattern with age (Saha et al. 2022), where the upper and lower arrows represent increasing and decreasing pattern changes with age, respectively.

FIGURE 2.

FNC component plots are displayed on the top, along with spatial maps depicting voxel‐wise GMV correlation with the loading parameters of the ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ are shown at the bottom for each component (asymmetric fusion). In the figure, we observe the voxel‐wise correlation for Components 2 and 4 have the highest positive (Component 2) and negative (Component 4) values.

FIGURE 3.

FNC components and spatial map of GMV components for the mCCA + jICA symmetric fusion approach. Component 3 from both the functional and structural data showing highly structured change patterns. In the figure, we observe increased functional connectivity between VS and SM domains or decreased functional connectivity between SM and CC correlated with ${\mathit{\text{SCPs}}}_{\mathit{sym}}$ indicating alterations in the bilateral sensorimotor cortex.

The results from the asymmetric fusion approach reveal noteworthy modularity in the fMRI result, suggesting structured changes occurring over the 2‐year period. And, interestingly, these were linked to strong and unique patterns of structural changes. In Figure 2, significant connections between ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ and voxel‐wise GMV data are evident. We also explored the covariation between the loading parameters. Notably, the correlation between loadings 4 and 2 of ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ is −0.38. In the spatial map figure, Component 2 displays a significant positive correlation (FDR‐corrected, p = 2.45x‐06), while the subcortical domain (SC) shows a widespread negative correlation with altered GMV data. The voxel‐wise correlation map suggests that Components 2 and 4 of ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ exhibit the same correlation with altered GMV data, but in opposite directions. Furthermore, a strong positive correlation (r = 0.35) between loadings 4 and 5 of ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ is observed. Similar to Component 4, our voxel‐wise correlation map reveals a significant negative correlation between ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ and voxel‐wise changed GMV data for Component 5 (FDR‐corrected, p = 9.83x‐04). Conversely, the SC shows a significant positive correlation, with notable differences in the subcortical region of the voxel‐wise correlation between Components 2 and 4. Component 4 demonstrates a higher voxel‐wise correlation in the SC.

The results from the symmetric fusion approach indicate that ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ associated with Components 2 and 3 exhibit significant changes with increasing age in the developing brain. Both components demonstrate an increasing trend with age, as evidenced by their negative T‐values. Component 3 shows increased brain functional connectivity between the visual (VS) and sensorimotor (SM) domains in the FNC data. Concurrently, there are decreasing changes in the bilateral sensorimotor cortex in the sMRI data over the 2‐year period. Moreover, the ${\mathit{FCP}}_{\mathit{sym}}$ of Component 3 exhibits a decreasing trend with age in functional connectivity between the VS and cerebellar (CB) domains, as well as between the SM and cognitive control (CC) domains. Furthermore, a two‐sample t‐test based on sex (biological sex assigned at birth) differences using the loading parameters from both modalities indicates that males show a smaller expression of change patterns in ${\mathit{SCP}}_{\mathit{sym}}$ for Component 2 compared to females.

3.2. Evaluation of Gender Showing Change Patterns Coupling

In the asymmetric multimodal fusion method, we have computed the difference (female–male) in the voxel‐wise correlation map between females and males to assess the gender effect. We found a positive correlation between the loading parameter of Component 2 and raw $\mathrm{\Delta }$GMV for both male and female data. Component 4 shows negative association with $\mathrm{\Delta }$GMV and significant correlation difference between males and females where females show higher voxel‐wise correlation than males as displayed in Figure 4. We conducted FDR correction on the correlation values with a significance threshold set at 0.001.

FIGURE 4.

Voxel‐wise correlations of females, males, and distinctions in voxel‐wise correlations map (female—male) between male and female loading parameters for ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ and raw $\mathrm{\Delta }$GMV. Females exhibit a stronger positive coupling for the ${\mathit{FCP}}_{\mathit{\text{asym}}}$ Component 2 and stronger negative coupling for the ${\mathit{FCP}}_{\mathit{\text{asym}}}$ Component 4 compared to males.

After applying the symmetric multimodal fusion technique, we computed the Pearson correlation between the loading parameters for all ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ and ${\mathit{\text{SCPs}}}_{\mathit{sym}}$, separately for males and females. This analysis aimed to explore gender differences in coupling, specifically the associations at the subject expression level. The variations in correlations between loading parameters of ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ and ${\mathit{\text{SCPs}}}_{\mathit{sym}}$, for males and females are visualized in Figure 5. Our findings indicate that females exhibited stronger coupling between ${\mathit{SCP}}_{\mathit{sym}}$, component 2 and ${\mathit{FCP}}_{\mathit{sym}}$ component 1 ($\mathrm{\Delta }$ r = 0.128, FDR‐corrected, p = 2.1895e‐11), ${\mathit{FCP}}_{\mathit{sym}}$ component 3 ($\mathrm{\Delta }$ r = 0.102, FDR‐corrected, p = 1.0081e‐07), and ${\mathit{FCP}}_{\mathit{sym}}$ component 4 ($\mathrm{\Delta }$ r = 0.111, FDR‐corrected, p = 6.7136e‐09) compared to males.

FIGURE 5.

Differences in correlations between male and female loading parameters for ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ and ${\mathit{\text{SCPs}}}_{\mathit{sym}}$. Females exhibit a stronger coupling between the ${\mathit{FCP}}_{\mathit{sym}}$ components 1, 3, and 4 and ${\mathit{SCP}}_{\mathit{sym}}$ Component 2 (indicated by *) expressions compared to males.

3.3. Association of Changes in Multivariate Pattern Coupling With Psychopathology Score, and Cognitive Score

We explored the relationship between changes in multivariate pattern coupling and subject measures, specifically psychopathology scores difference and cognitive scores difference. Our results reveal that both Components 4 and 5 exhibit a significant negative association with raw $\mathrm{\Delta }$GMV data within the lower quartile of psychiatric scores, as illustrated in Figure 6. The voxel‐wise correlation maps for cognitive scores are presented in Figure 7. Component 2 of ${\mathit{FCP}}_{\mathit{\text{asym}}}$, predominantly shows a positive correlation with the $\mathrm{\Delta }$GMV data within the lower quartile of cognitive scores. Regarding Component 5, it displays negative correlations in the lower quartiles. We observed a noteworthy difference between the upper and lower quartiles for both psychopathology and cognitive scores. Furthermore, we applied FDR correction at a significance level of 0.05 to the correlation values and only significant results are shown in the figures.

FIGURE 6.

Voxel‐wise correlation difference map (upper–lower) illustrating the relationship between loading parameters of ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$, and raw $\mathrm{\Delta }$GMV for the upper and lower quartile of psychiatric scores. Components 4 and 5 exhibit notably stronger voxel‐wise correlations within the lower quartile of psychiatric scores (differences are negative).

FIGURE 7.

Voxel‐wise correlation difference map (upper–lower) illustrating the relationship between loading parameters of ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ and raw $\mathrm{\Delta }$GMV for the upper and lower quartile of cognitive scores. Components 2 and 5 exhibit notably stronger voxel‐wise correlations within the lower quartile of cognitive scores.

4. Discussion

To the best of our knowledge, this is the first study to estimate the multivariate pattern coupling/covariations in structural and functional connectivity associated with age progression using a very large sample of nearly 2734 subjects in developing brain. Our study introduces two data‐driven multivariate multimodal fusion analyses to delineate change patterns in brain structure and function among healthy adolescents over a 2‐year period. The key findings include (1) identification of intriguing patterns in FNC and GMV data, demonstrating links to individual variations. Specifically, ${\mathit{FCP}}_{\mathit{\text{asym}}}$ components 2 and 4 display significant associations between their loadings and voxel‐wise GMV data, (2) discovery of structured ${\mathit{\text{FCPs}}}_{\mathit{sym}}$, such as enhanced connectivity between visual and sensorimotor domains or decreasing coupling trend between the SM and CC, correlated with ${\mathit{\text{SCPs}}}_{\mathit{sym}}$ indicating alterations in the bilateral sensorimotor cortex, (3) observation of stronger coupling between brain functional and structural changes in females compared to males, unveiling gender‐related disparities, (4) recognition of multiple functional connectivity and gray matter volume data linked to longitudinal changes in psychopathology and cognition scores within the developing brain.

Understanding longitudinal changes in the developing brain is essential for comprehending cognitive development and neural maturation. Multimodal fusion of functional connectivity and sMRI provides a powerful approach to examining these complex developmental processes. This study highlights the intricate relationship between functional and structural changes over time. Using multimodal fusion, significant connections between ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$ and voxel‐wise GMV have been identified, supporting previous findings on multimodal imaging in brain maturation (Cao et al. 2017; Douaud et al. 2014). Asymmetric fusion reveals modularity trends over 2 years, suggesting structured developmental changes occurring during brain development (Smith et al. 2011). These findings further reinforce the interplay between functional and structural maturation, aligning with prior research linking functional connectivity to anatomical substrates (Damoiseaux and Greicius 2009; Honey et al. 2009; Vázquez‐Rodríguez et al. 2019). The study identified intriguing relationships, such as a− 0.38 correlation between loadings 4 and 2, revealing potential opposing effects in functional networks (Fjell et al. 2013). Components 2 and 4 show similar yet opposing correlations with GMV changes, highlighting region‐specific developmental patterns (Sheline et al. 2009). Spatial mapping underscores these variations, particularly in the SC domain, where contrasting correlations emphasize the significance of regional differences in brain maturation. These findings enrich our understanding of how specific brain regions follow distinct developmental trajectories, influencing cognitive and behavioral outcomes. These insights contribute to understanding how variations in brain development influence cognitive and behavioral outcomes.

The symmetric multimodal fusion approach, mCCA + jICA, reveals complex relationships between structural and functional changes over time. Previous studies (Alexander‐Bloch et al. 2013; Bassett et al. 2008; Hagmann et al. 2008; Honey et al. 2009; Michael et al. 2010; Olesen et al. 2003; Rykhlevskaia et al. 2008; Segall et al. 2012; Skudlarski et al. 2010; Van Den Heuvel et al. 2009; Yu et al. 2011) demonstrate clear associations between structural and functional connectivity, highlighting covarying patterns in longitudinal changes. These findings emphasize the interplay between structural alterations and functional connectivity in brain maturation, consistent with research on the mutual influence of structure and function during development (Douaud et al. 2014; Hermundstad et al. 2013). Our analysis shows that Component 2 and 3's ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ exhibit negative T‐values, indicating an age‐related increase in adolescents. Component 3 reveals enhanced functional connectivity between the VS and SM domains, critical for integrating visual and motor information to support perception, action, and cognition (Kaufmann et al. 2015). This evidence suggests that the developing brain forms connections among various regions while adapting to external stimuli, which likely play a critical role in distinguishing between regular and irregular neural dynamics. The enhanced connectivity between visual and sensorimotor domains aligns with prior research on their dynamic interconnectivity (Dayan and Cohen 2011; Ungerleider and Haxby 1994), highlighting the role of visual inputs in motor coordination and sensorimotor integration. We observed decreased functional coupling between the VS‐CB and SM‐CC regions with age, indicated by the highest positive T‐values, suggesting neural network reorganization. The correlation between ${\mathit{FCP}}_{\mathit{sym}}$ and ${\mathit{SCP}}_{\mathit{sym}}$ alterations, particularly in the bilateral sensorimotor cortex, underscores the link between functional connectivity and structural modifications. These findings reveal covarying patterns of structural and functional dynamics during development, emphasizing their close relationship (Hagmann et al. 2008). Studies also suggest that changes in neuronal functioning correspond to alterations in gray matter structure (Schultz et al. 2012). Overall, understanding how structural changes covary with functional connectivity offers critical insights into the coordinated and interdependent mechanisms driving brain maturation (Dennis et al. 2013; Supekar et al. 2010).

In this study, we employed both asymmetric and symmetric multimodal fusion methods to explore gender effects on functional and structural change coupling, aiming to discern nuanced differences in neural dynamics between males and females. In the asymmetric multimodal fusion method, the examination of voxel‐wise correlation maps revealed intriguing gender effects. Specifically, a positive correlation between the loading parameter of Component 2 and raw $\mathrm{\Delta }$GMV was observed for both males and females. This finding is in line with previous studies indicating gender‐related differences in brain structure and connectivity, suggesting that both gender exhibit similar patterns of association between specific components and structural changes (Ingalhalikar et al. 2014). However, Component 4 exhibited a gender‐specific association with $\mathrm{\Delta }$GMV, demonstrating a significant correlation difference between males and females. Females exhibited higher voxel‐wise correlation than males, suggesting a nuanced gender‐related divergence in the relationship between this specific ${\mathit{FCP}}_{\mathit{\text{asym}}}$ and structural changes. Gender‐specific differences in brain structure and connectivity have been previously reported (Ruigrok et al. 2014), and the present study adds granularity by highlighting component‐specific distinctions. The application of the symmetric multimodal fusion technique further delved into gender differences in coupling at the subject expression level. The results revealed distinct variations in coupling patterns between males and females, shedding light on gender‐dependent associations within the neural network. Notably, females exhibited stronger coupling between ${\mathit{SCP}}_{\mathit{sym}}$ component 2 and ${\mathit{FCP}}_{\mathit{sym}}$ components 1, 3, and 4 compared to males. This discovery indicates that women who play a major role in the structural change pattern (${\mathit{SCP}}_{\mathit{sym}}$ component 2) also make substantial contributions to the ${\mathit{\text{FCPs}}}_{\mathit{sym}}$ of Components 1, 3, and 4. These observations suggest gender‐dependent variations in the interplay between structural changes and functional connectivity, emphasizing the importance of considering individual differences in understanding brain dynamics (Ingalhalikar et al. 2014; Ritchie et al. 2018). Based on the correlation values we have, it can be inferred that females demonstrate a stronger association between the function and structure of the developing brain than males.

Our analysis uncovered several functional and structural connectivity change pattern coupling associated with longitudinal changes in psychopathology and cognition scores within the developing brain. Specifically, we observed a significantly higher negative correlation between ${\mathit{FCP}}_{\mathit{\text{asym}}}$ components 4 and 5 and raw $\mathrm{\Delta }$GMV in subjects with low psychiatric scores compared to those with high psychiatric scores. This finding implies that individuals with lower psychiatric scores demonstrate a stronger association between ${\mathit{\text{FCPs}}}_{\mathit{\text{asym}}}$, such as increased brain functional coupling between the VS‐SM and CB‐SC, or a decrease in change patterns between VS‐CB and SM‐SC, and raw $\mathrm{\Delta }$GMV. Additionally, ${\mathit{FCP}}_{\mathit{\text{asym}}}$ components 2 and 5 exhibit positive and negative associations, respectively, with the $\mathrm{\Delta }$GMV data within the lower quartile of cognitive scores. This indicates that individuals with lower cognitive scores show both positive (Component 2) and negative (Component 5) voxel‐wise correlations. Notably, a significant higher correlation is observed between both ${\mathit{FCP}}_{\mathit{\text{asym}}}$ components 2 and 5 and raw $\mathrm{\Delta }$GMV within the lower quartile of cognitive scores compared to the upper quartile. This suggests that individuals contributing less to changes in cognitive scores from baseline to 2 years exhibit a stronger association between functional and structural data in the developing brain. These findings align with recent studies that have reported age‐related changes in vasculature, brain anatomy, and brain function collectively contributing to complex interactions that influence cognitive alterations (Fabiani et al. 2021; Zimmerman et al. 2021). We also analyzed the impact of site on our analysis by running the model with and without site nuisance regressors. The results showed no significant influence of site information on our findings.

In this manuscript, we present an innovative method that combines various data types to analyze patterns across multiple domains. We apply this method to explore the connection between brain structure and function using a combination of FNC‐sMRI data. Several promising research directions are apparent for further exploration. For instance, expanding the use of additional imaging techniques could enhance our understanding of overall brain change patterns, a path we aim to explore in future investigations. Furthermore, future research should delve into how increased component numbers affect these couplings, where varying the component count may reveal stronger connections between different brain regions. Our proposed methods also have certain constraints that warrant consideration. First, it is important to note that while mCCA + jICA operates on ICA components instead of the original imaging data (e.g., using 3D contrast images instead of 4D fMRI data), this approach may lose some temporal information. However, working with these ICA components presents advantages such as reduced dimensionality (Calhoun and Adali 2008) and a simplified space for linking the data (Smith et al. 2009). Second, our utilization of ICA assumes linearity in capturing brain functional pattern changes. Nonetheless, recent research by Motlaghian et al. highlights the potential presence of nonlinear relationships within functional networks, a factor often overlooked in linear analyses (Motlaghian et al. 2022). Exploring nonlinear methodologies could offer valuable insights into age‐related functional change patterns in the developing brain. Finally, this study utilized data from youths aged 9–10 years, potentially indicating relatively low levels of psychiatric symptomatology among the subjects (Escrichs et al. 2021). However, it is anticipated that their psychopathology load may increase during adolescence (Paus et al. 2008). This relationship between brain and behavior might evolve, growing stronger, weaker, or altering in some other way, which could be directly examined in future waves of longitudinal ABCD data. This would enable the formulation of clear hypotheses.

5. Conclusion

In this study, we propose asymmetric and symmetric multimodal fusion techniques to examine longitudinal brain changes in adolescents. These methods explore structural and functional changes and their correlations with aging using FNC matrices and GMV data from the ABCD dataset. Our asymmetric fusion results reveal significant associations between FCP components 2 and 4 and voxel‐wise GMV, suggesting that multivariate FNC and GMV patterns relate to aging. The symmetric fusion technique identifies structured ${\mathit{\text{FCPs}}}_{\mathit{sym}}$, such as increased visual‐sensorimotor connectivity and decreased sensorimotor‐cognitive control connectivity, correlated with ${\mathit{\text{SCPs}}}_{\mathit{sym}}$ in the bilateral sensorimotor cortex. Studying functional connectivity–GMV coupling over time helps reveal how the developing brain modulates its structural and functional relationships for efficient processing. Our findings highlight brain components contributing to gender‐based differences and reveal functional–structural couplings linked to psychopathology and cognition. To our knowledge, our proposed methodologies for investigating age‐related change patterns and coupling in the developing brain is the first approach to assess whole‐brain covarying functional and structural changes in longitudinal data.

Author Contributions

The study was designed by R.S. and V.D.C. Data preprocessing was performed by Z.F. The experimental design was contributed by R.S. and V.D.C. R.S. conducted the analyses, performed the statistical analysis, and drafted the initial manuscript. The manuscript was edited by V.D.C., D.K.S., Z.F., M.D., and R.F.S. All authors participated in various stages of experimental analyses and approved the final version of the manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Data S1.

Saha, R. , Saha D. K., Fu Z., et al. 2025. “Analysis of Longitudinal Change Patterns in Developing Brain Using Functional and Structural Magnetic Resonance Imaging via Multimodal Fusion.” Human Brain Mapping 46, no. 10: e70241. 10.1002/hbm.70241.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.