A graph transformer-based foundation model for brain functional connectivity network

Abstract

Although foundation models have advanced many medical imaging fields, their absence in neuroimage analysis limits progress in neuroscience and clinical practice. Brain functional connectivity (FC) analysis is central to understanding brain function and widely used in neuroscience. We propose a foundation model tailored for brain functional connectivity networks (FCN). Our graph transformer model integrates node and edge embeddings to extract FCN features and adapts flexibly to classification, regression, and clustering via task-specific adapters. We validate the model on fMRI data from 10,718 subjects across multiple tasks: gender classification, mental disorder classification (distinguishing schizophrenia or autism from healthy population), brain age prediction, and depressive and anxiety disorder biotyping. Compared to 14 competing methods, our model consistently outperforms them. Moreover, it facilitates biomarker discovery by identifying task-specific FC patterns. In summary, we present a novel, versatile foundation model for FCN that advances neuroimaging research through scalable and interpretable analysis.

1. Introduction

Foundation models have achieved success in medical applications, including cancer diagnosis from images [1], medical question answering and report generation [2], as well as segmentation and enhancement of medical images [3]. These models leverage large-scale data to capture complex patterns, improving diagnostic accuracy and clinical decision-making across multiple domains.

In neuroscience, brain functional connectivity (FC), which reflects the interactions between different brain regions, has become a crucial tool in classifying various brain disorders [4, 5]. Advanced graph and hypergraph models have been developed to capture these intricate dependencies, facilitating the identification of neuropathological biomarkers [6, 7]. Furthermore, preliminary pretrained models and transfer learning approaches [8, 9] have shown promising results in enhancing the accuracy of diagnostic tasks, demonstrating their potential for more effective disease detection and prognosis prediction.

Foundation models have been developed for tasks such as brain disease diagnosis. For instance, BrainMass [10] builds a foundational model through self-supervised pretraining on brain imaging datasets, capturing complex brain network patterns for effective disease diagnosis. Similarly, HGFM [11] utilizes multidimensional pretraining to uncover high-order correlation patterns across diverse brain disease datasets. However, existing methods are typically designed for specific tasks, limiting their applicability across diverse clinical scenarios and hindering efficient clinical translation.

Developing an effective foundation model for functional connectivity networks (FCN) faces key challenges. First, representing both node-specific features and edge-level interactions accurately is nontrivial. Second, adapting pretrained models to diverse and clinically relevant downstream tasks requires flexible architectures that can generalize well across heterogeneous datasets [12]. In this paper, we propose a foundation model for FCN analysis with a novel graph-transformer that captures complex node-edge relationships. Task-specific decoders enable adaptation across various downstream tasks, ensuring strong performance.

Our study makes three main contributions.

1. We develop a foundation model for brain FCN analysis. It uses a graph transformer-based autoencoder that jointly encodes node and edge embeddings to capture complex connectivity patterns.

2. We design a flexible framework that integrates task-specific decoders, enabling efficient transfer learning for diverse clinical brain network analysis tasks, including gender classification, mental disorder diagnosis, brain age prediction, and biotype identification.

3. We validate the effectiveness and interpretability of our model on nine large-scale datasets totaling 10,718 scans, demonstrating improved performance across multiple challenging tasks and illustrating its potential for reliable clinical application and biomarker discovery.

2. Related works

This section is divided into two parts. Section 2.1 reviews graph transformers and their applications, while Section 2.2 summarizes graph autoencoder architectures and their uses.

2.1. Graph transformer

Since transformers were introduced, diverse adaptations have advanced graph data processing. Graph transformers mainly follow two approaches: compositional and embedded.

Compositional methods combine graph neural networks (GNNs) with transformers directly. This simple yet efficient strategy is seen in GraphTrans [13] and SGFormer [14]. In neuroscience, SCDA [8] uses pretrained GNNs and transformers for efficient knowledge transfer without source data. While compositional methods are widely adopted for their simplicity and effectiveness, their shallow integration of GNNs and transformers limits performance in modeling complex graph dependencies.

Embedded methods embed graph computations within attention mechanisms to extract deeper features, as seen in HAMLET [15] and GT [16]. In neuroscience, BrainNetTF [17] exposes hidden similarities among brain nodes through orthogonal clustering. Cai et al. [18] proposed a graph transformer framework that combines structural MRI (sMRI) and diffusion MRI (dMRI) to enhance brain age estimation and Alzheimer’s diagnosis. These methods show great promise but tend to be computationally intensive, increasing complexity. Their focus on deep features may reduce interpretability, limiting their practical use where clear explanations are needed.

In this study, we propose a novel graph transformer that deeply integrates graph computations to jointly capture both node- and edge-level embeddings in brain networks, while improving interpretability. This method supports diverse downstream tasks effectively.

2.2. Graph autoencoder

Graph autoencoders (GAEs) have made significant progress, helping reveal complex community structures [19]. Research mainly follows three approaches: decoding edge features, decoding node features, or decoding both together.

Edge feature decoding often uses inner product methods, offering simple and efficient reconstruction. This is common in methods like GAE [20], VGAE [20], and DAEGC [21]. However, these methods may miss subtle graph relationships due to their simplicity. Node feature decoding uses multilayer perceptrons (MLP) or GNNs, such as GALA [22] and GCN-BSD [23]. This better guides feature embedding and improves performance. Yet, focusing on nodes alone limits capturing edge dependencies and the overall graph structure. Combining both approaches, models like DGAE [24] and MCGMAE [25] decode node and edge features simultaneously by leveraging latent node representations. While this decoding approach improves the representation of both node and edge features, it still faces challenges in capturing complex interdependencies between nodes and edges at multiple levels.

These advances show GAEs’ potential but reveal a key limitation: poor modeling of latent edge representations, causing suboptimal learning of edge relationships. In FCNs, nodes (brain regions) and edges (FCs) are both essential for reflecting neural processes. Traditional methods often fail to integrate them effectively, missing complex interactions. To overcome these issues, we integrate a graph transformer into the autoencoder. Using transformer’s attention mechanism, it captures complex node-edge relationships and adapts well to various tasks. This method improves feature representation and enables robust graph learning, especially for complex neural networks.

3. Methodology

Fig. 1 outlines our method’s two steps: pretraining and fine-tuning. In pretraining (Fig. 1(a)), we train the model on large-scale UK Biobank functional MRI (fMRI) data using a graph autoencoder. Then (Fig. 1(b)), the pretrained model is fine-tuned on task-specific data. For supervised tasks, a task decoder adapter handles decoding; for unsupervised tasks, the graph autoencoder decodes directly. The model architecture for both stages is shown in Fig. 1(c), featuring a graph transformer-based encoder and decoder that jointly process node and edge features. Fig. 1(d) depicts the task decoder, where a task token added to input enables attention weights to capture node and edge contributions, improving performance and interpretability. Key symbols used throughout are summarized in Table 1, and common abbreviations are listed in Table S1 of the Supplementary Materials.

Fig. 1.

The framework of our proposed FCN foundation model. (a) Pretraining. (b) Fine-tuning. (c) Pretraining and fine-tuning model. (d) Supervised downstream task. The solid arrows represent the flow of data, while the dashed arrows point to the detailed structural information.

Table 1.

Symbols and descriptions.

Symbols	Descriptions
$V=\left\{v_1,v_2,\dots ,v_N\right\}$	set of nodes
$V_{\mathit{\text{cls}}}=\left\{v_1,v_2,\dots ,v_N,v_{\mathit{\text{cls}}}\right\}$	extended set of nodes incorporating a classification token
$V_{\mathit{\text{reg}}}=\left\{v_1,v_2,\dots ,v_N,v_{\mathit{\text{reg}}}\right\}$	extended set of nodes incorporating a regression token
$A$	original adjacency matrix
$\hat{A}$	reconstructed adjacency matrix through decoding process
$A^{\prime }$	latent embedding representation of adjacency matrix
$A_{\mathit{\text{hidden}}}^l$	adjacency matrix derived from self-attention at layer $l$
$A_{\mathit{\text{task}}}^{\prime }$	task-specific adjacency matrix
$A_{\mathit{\text{cls}}}^{\prime }$	classification-decoder-specific adjacency matrix input
$A_{\mathit{\text{reg}}}^{\prime }$	regression-decoder-specific adjacency matrix input
$X$	original node feature matrix
$\widehat{X}$	reconstructed node feature matrix through decoding
$X^{\prime }$	latent embedding representation of node feature matrix
$X_{\mathit{\text{hidden}}}^l$	node feature matrix derived from self-attention at layer $l$
$X_{\mathit{\text{task}}}^l$	task-specific node feature matrix
$X_{\mathit{\text{cls}}}^{\prime }=\left[X^{\prime },x_{\mathit{\text{cls}}}^{\prime }\right]$	classification-decoder-specific node feature matrix input
$X_{\mathit{\text{cls}}}^{\prime \prime }=\left[X^{\prime \prime },x_{\mathit{\text{cls}}}^{\prime \prime }\right]$	classification-decoder-specific node feature matrix output
$X_{reg}^{\prime }=\left[X^{\prime },x_{reg}^{\prime }\right]$	regression -decoder-specific node feature matrix input
$X_{reg}^{\prime \prime }=\left[X^{\prime \prime },x_{reg}^{\prime \prime }\right]$	regression -decoder-specific node feature matrix output
$x_{\mathit{\text{clu}}}^{\prime }$	feature representation for clustering task input
${\mathit{\text{Attn}}}^l$	attention weight matrix at layer $l$
${\mathit{\text{Attn}}}_{\mathit{\text{cls}}}$	classification-specific attention weight matrix
$c_k$	centroid vector of cluster $k$
$q_{\mathit{\text{clu}},k}$	soft assignment probability of $x_{\mathit{\text{clu}}}^{\prime }$ to cluster $k$

3.1. Model pretraining

3.1.1. The edge-node encoder block and edge-node decoder block

We model the FCN as a graph $G=(V,A,X)$, where $V=\left\{v_1,v_2,\dots ,v_N\right\}$ are $N$ brain nodes, $A\in \{\mathrm{0,\; 1}{\}}^{N\times N}$ is the adjacency matrix representing edges, and $X\in {\mathbb{R}}^{N\times d}$ is the node feature matrix. The raw FC matrix, computed from the Pearson correlation coefficients between functional networks, is used to derive $X$, which has $d=N$ features per node. To focus on biologically meaningful edges, the original FC matrix is sparsified by retaining only the top $H\%$ strongest connections, resulting in the adjacency matrix $A$. This process highlights both key node connections and unique node characteristics.

To embed node and edge features simultaneously, we design an edge-node encoder with $L$ layers of graph transformers (Fig. 1(c)). For the $l$-th layer ($l<L$), the self-attention is computed as:

$$Att{n}^l=\text{softmax}\left(\frac{X^l{W}_q^l{\left(X^l{W}_k^l\right)}^T}{\sqrt{d_k}}+A^l{W}_e^l\right)$$ (1)

$$X_{\mathit{\text{hidden}}}^l=\text{BatchNorm}\left(X^l+At{tn}^l{X}^l{W}_v^l\right)$$ (2)

$$A_{\mathit{\text{hidden}}}^l=\text{BatchNorm}\left(A^l+At{tn}^l{A}^l{W}_e^l\right)$$ (3)

where $W_q^l,W_k^l,W_v^l\in {\mathbb{R}}^{d\times d_k}$ are node feature projection matrices, and $d_k$ is the hidden layer feature dimension. $W_e^l\in {\mathbb{R}}^{N\times N}$ projects adjacency features. ${\mathit{\text{Attn}}}^l\in {\mathbb{R}}^{N\times N}$ is the attention weight matrix. The next layer inputs are:

$$X^{l+1}=\left(\text{BatchNorm}\left(X_{\mathit{\text{hidden}}}^l+\text{ReLU}\left(X_{\mathit{\text{hidden}}}^l{W}_{X,1}^l\right)W_{X,2}^l\right)\right)W_{X,3}^l$$ (4)

$$A^{l+1}=\text{BatchNorm}\left(A_{\mathit{\text{hidden}}}^l+\text{ReLU}\left(A_{\mathit{\text{hidden}}}^l{W}_{A,1}^l\right)W_{A,2}^l\right)$$ (5)

where $W_{X,1}^l,W_{X,2}^l,W_{X,3}^l$ and $W_{A,1}^l,W_{A,2}^l$ are linear layers in the feedforward networks for nodes and edges, respectively. After $L$ layers, the embedded node features $X^{\prime }\in {\mathbb{R}}^{N\times m}$ and the adjacency embeddings $A^{\prime }\in {\mathbb{R}}^{N\times N}$ are obtained, where $m$ is the embedding dimension. These embeddings, $X^{\prime }$ and $A^{\prime }$, can generate task-specific representations $G_{\mathit{\text{task}}}=\left(V_{\mathit{\text{task}}},A_{\mathit{\text{task}}}^{\prime },X_{\mathit{\text{task}}}^{\prime }\right)$ for downstream brain network tasks.

The edge-node decoder is similar to the edge-node encoder with $L$ layers of graph transformers, processing embeddings $X^{\prime }$ and $A^{\prime }$. Using the same operations as described in equations (1) to (5), the decoder outputs reconstructed node features $\hat{X}$ and adjacency matrix $\hat{A}$. These reconstructions enable graph recovery for downstream applications.

3.1.2. Loss function design

The pretraining model uses a hybrid loss to optimize node and edge embeddings simultaneously, improving graph representation quality. The loss is defined as:

$${ℒ}_{\mathit{\text{pre}}}={\lambda }_1{ℒ}_{\mathit{\text{adj}}}+{\lambda }_2{ℒ}_{\mathit{\text{node}}}$$ (6)

where ${\lambda }_1$ and ${\lambda }_2$ are weights.

${ℒ}_{\mathit{\text{adj}}}$ represents the binary cross-entropy loss [20] employed to optimize the reconstruction of the adjacency matrix. To address class imbalance, we weight positive edges by $w_{\mathit{\text{pos}}}=\frac{\text{count}\left(A_i=0\right)}{\text{count}\left(A_i=1\right)}$, where $A_i$ represents an element of $A$, and $\text{count}\left(A_i=0\right)$ and $\text{count}\left(A_i=1\right)$ are counts of zeros and ones in $A$. Thus, ${ℒ}_{\mathit{\text{adj}}}$ can be expressed as:

$${ℒ}_{adj}\left(\hat{A},A,w_{pos}\right)=-\frac{1}{N^2}\sum _{i=1}^{N^2}\left(A_i{w}_{pos}\text{log}\left({\hat{A}}_i\right)+\left(1-A_i\right)\text{log}\left(1-{\hat{A}}_i\right)\right)$$ (7)

where ${\hat{A}}_i$ represents an element of $\hat{A}$.

${ℒ}_{\mathit{\text{node}}}$ is the loss for reconstructing the node feature matrix, expressed as:

$${ℒ}_{\mathit{\text{node}}}(\hat{X},X)=\frac{1}{2}\Vert X-\hat{X}{\Vert }_F^2$$ (8)

where $\Vert \cdot {\Vert }_F$ is the Frobenius norm [24]. These losses complement each other: ${ℒ}_{\mathit{\text{adj}}}$ focuses on graph structure, while ${ℒ}_{\mathit{\text{node}}}$ ensures accurate node features. Together, they yield a more complete graph representation.

3.2. Model fine-tuning for classification

3.2.1. The classification decoder block

The fine-tuning pipeline for classification is illustrated in Fig. 1(d). We add a learnable classification token $x_{\text{cls}}^{\prime }$ to the node features $X^{\prime }$, and a corresponding token node $v_{\text{cls}}$ to the adjacency matrix $A^{\prime }$. This defines the graph $G_{\text{task}}$ for classification as $G_{cls}=\left(V_{cls},A_{cls}^{\prime },X_{cls}^{\prime }\right)$, where $V_{cls}=\left\{v_1,v_2,\dots ,v_N,v_{cls}\right\}$ includes $N+1$ nodes. The adjacency matrix is $A_{\mathit{\text{cls}}}^{\prime }\in \{\mathrm{0,\; 1}{\}}^{(N+1)\times (N+1)}$, and the feature matrix is $X_{\text{cls}}^{\prime }=\left[X^{\prime },x_{\mathit{\text{cls}}}^{\prime }\right]\in {\mathbb{R}}^{(N+1)\times m}$. The classification decoder uses a self-attention mechanism:

$${\mathit{\text{Attn}}}_{cls}=\text{softmax}\left(\frac{X_{cls}^{\prime }{W}_q{\left(X_{cls}^{\prime }{W}_k\right)}^T}{\sqrt{d_k}}+A_{cls}^{\prime }{W}_e\right)$$ (9)

$$X_{cls}^{″}={\mathit{\text{Attn}}}_{cls}{X}_{cls}^{\prime }{W}_v$$ (10)

where $W_q,W_k,W_v\in {\mathbb{R}}^{m\times d_k}$ are linear projections for node features, and $W_e\in {\mathbb{R}}^{(N+1)\times (N+1)}$ for adjacency. ${\mathit{\text{Attn}}}_{\mathit{\text{cls}}}\in {\mathbb{R}}^{(N+1)\times (N+1)}$ is the attention weight matrix for the classification task. The output embedding is $X_{\mathit{\text{cls}}}^{″}=\left[X^{″},x_{\mathit{\text{cls}}}^{″}\right]\in {\mathbb{R}}^{(N+1)\times m}$. Finally, $x_{\text{cls}}^{″}$ is passed through an MLP to predict the classification probability $\hat{y}$.

3.2.2. Loss function design

For classification, the cross-entropy loss [11] ${ℒ}_{ce}$ is used to optimize the model:

$${ℒ}_{ce}=-\frac{1}{M}\sum _{i=1}^M\sum _{j=1}^C{y}_{i,j}\text{log}\left({\hat{y}}_{i,j}\right)$$ (11)

where $M$ is the number of subjects, $C$ is the number of classes. For subject $i,{\hat{y}}_{i,j}$ is the predicted label, and $y_{i,j}$ is the ground truth label. Therefore, the total loss can be expressed as:

$${ℒ}_{\mathit{\text{total}}}={\lambda }_1{ℒ}_{\mathit{\text{adj,cls}}}+{\lambda }_2{ℒ}_{\mathit{\text{node,cls}}}+{\lambda }_3{ℒ}_{\mathit{\text{ce}}}$$ (12)

where ${ℒ}_{\mathit{\text{adj,cls}}}$ and ${ℒ}_{\mathit{\text{node,cls}}}$ are the adjacency and node reconstruction losses on fine-tuning data, computed as described in equations (7) and (8). ${\lambda }_i(i=\mathrm{1,\; 2,\; 3})$ are weights. During fine-tuning, graph embedding and classification are jointly optimized in a unified framework [24].

3.3. Model fine-tuning for regression

3.3.1. The regression decoder block

The fine-tuning pipeline for regression (Fig. 1(d)) is similar to the classification task. We add a learnable regression token $x_{\mathit{\text{reg}}}^{\prime }$ to the features $X^{\prime }$ and a token node $v_{\mathit{\text{reg}}}$ to the adjacency $A^{\prime }$. The graph $G_{\mathit{\text{task}}}$ for regression is represented as $G_{\mathit{\text{reg}}}=\left(V_{\mathit{\text{reg}}},A_{\mathit{\text{reg}}}^{\prime },X_{\mathit{\text{reg}}}^{\prime }\right)$, where $V_{\mathit{\text{reg}}}=\left\{v_1,v_2,\dots ,v_N,v_{\mathit{\text{reg}}}\right\}$ includes $N+1$ nodes. The adjacency matrix is $A_{\mathit{\text{reg}}}^{\prime }\in \{\mathrm{0,1}{\}}^{(N+1)\times (N+1)}$, and the feature matrix $X_{\mathit{\text{reg}}}^{\prime }=\left[X^{\prime },x_{reg}^{\prime }\right]\in {\mathbb{R}}^{(N+1)\times m}$. The output $X_{reg}^{″}=\left[X^{″},x_{reg}^{″}\right]\in {\mathbb{R}}^{(N+1)\times m}$ is computed in a manner similar to that described in equations (9) and (10). The vector $x_{reg}^{″}$ is passed to an MLP to predict the regression output $\hat{y}$.

3.3.2. Loss function design

For regression, we use the mean absolute error (MAE) loss [26]:

$${ℒ}_{\mathit{\text{mae}}}=\frac{1}{M}\sum _{i=1}^M\left|{\hat{y}}_i-y_i\right|$$ (13)

where, for subject $i,{\hat{y}}_i$ is the predicted label, and $y_i$ is the ground truth label. The total loss combines embedding and regression losses:

$${ℒ}_{\mathit{\text{total}}}={\lambda }_1{ℒ}_{\mathit{\text{adj,reg}}}+{\lambda }_2{ℒ}_{\mathit{\text{node,reg}}}+{\lambda }_3{ℒ}_{\mathit{\text{mae}}}$$ (14)

where ${ℒ}_{\mathit{\text{adj,reg}}}$ and ${ℒ}_{\mathit{\text{node,reg}}}$ are adjacency and node feature losses, calculated as described in equations (7) and (8). Both embedding and regression tasks are jointly optimized in a unified framework.

3.4. Model fine-tuning for clustering

3.4.1. The clustering decoder block

For clustering, the fine-tuned model performs unsupervised clustering directly. The embedded node feature matrix $X_{\mathit{\text{task}}}^{\prime }\in {\mathbb{R}}^{N\times m}$ is flattened into a vector $x_{\mathit{\text{clu}}}^{\prime }\in {\mathbb{R}}^h$, where $h=N\times m$. This vector is processed by the clustering decoder. We compute the squared Euclidean distance between $x_{clu}^{\prime }$ and each cluster center $c_k$:

$${‖x_{clu}^{\prime }-c_k‖}^2=\sum _{j=1}^h{\left(x_{clu,j}^{\prime }-c_{k,j}\right)}^2$$ (15)

where $x_{clu,j}^{\prime }$ denotes the $j$-th feature value of $x_{clu}^{\prime }$, and $c_{k,j}$ represents the $j$-th feature value of the cluster $k$ center $c_k$, where $k\in \{1,\dots ,K\}$. The probability that a subject belongs to cluster $k$ is:

$$q_{clu,k}=\frac{{\left(1+{‖x_{clu}^{\prime }-c_k‖}^2\right)}^{-1}}{\sum _{k=1}^K{\left(1+{‖x_{clu}^{\prime }-c_k‖}^2\right)}^{-1}}$$ (16)

Therefore, the probability that the subject $i$ belongs to cluster $k$ is $q_{i,k}$.

3.4.2. Loss function design

For clustering, we optimize the model using the Kullback-Leibler (KL) divergence loss [24] ${ℒ}_{kl}$:

$${ℒ}_{kl}=\frac{1}{M}\sum _{i=1}^M\sum _{k=1}^K{p}_{i,k}log\frac{p_{i,k}}{q_{i,k}}$$ (17)

where the target distribution $p_{i,k}$ is defined as:

$$p_{i,k}=\frac{q_{i,k}^2/\sum _{i=1}^M{q}_{i,k}}{\sum _{k=1}^K{q}_{i,k}^2/\sum _{i=1}^M{q}_{i,k}}$$ (18)

The total loss combines clustering and graph embedding losses:

$${ℒ}_{\mathit{\text{total}}}={\lambda }_1{ℒ}_{\mathit{\text{adj,clu}}}+{\lambda }_2{ℒ}_{\mathit{\text{node,clu}}}+{\lambda }_3{ℒ}_{kl}$$ (19)

where ${ℒ}_{\mathit{\text{adj,clu}}}$ and ${ℒ}_{\mathit{\text{node,clu}}}$ are embedding losses computed in a manner similar to that defined in equations (7) and (8). Both clustering and embedding are jointly optimized in a unified framework.

4. Experiments

4.1. Pretraining setup

4.1.1. Data acquisition and FC calculation

We used fMRI data from the UK Biobank project [27] (Project 34175, PI: Yuhui Du) for model pretraining. Participants with mental health disorders, neurological conditions, or diseases affecting brain function were excluded, resulting in 6,694 participants. Demographic details and ethical approval information are available in the Supplementary Materials. Data preprocessing involved discarding the first 10 volumes, slice-timing correction, motion correction, normalization to MNI space, resampling to 3×3×3 mm³ voxels, and Gaussian smoothing (6 mm FWHM).

To define consistent brain nodes across subjects, we applied the NeuroMark ICA method [28], which extracts 53 functional networks per subject based on reliable network templates. This method ensures component independence for node accuracy and correspondence across subjects for aligned brain nodes [29]. NeuroMark is publicly available at www.yuhuidu.com and has been used to study functional network biomarkers of psychiatric disorders [30, 31]. The 53 brain nodes were categorized into seven functional domains: subcortical (SC), auditory (AU), sensorimotor (SM), visual (VI), cognitive control (CC), default mode (DM), and cerebellar (CB) (see Table S4). FC matrices were calculated by Pearson correlation between paired nodes’ time series.

4.1.2. Implementation details

To ensure unbiased evaluation, we used nested 5-fold cross-validation. In the outer loop, the dataset was divided into five folds, with each training set undergoing a separate 5-fold cross-validation for hyperparameter optimization in the inner loop, where a grid search evaluated hyperparameter combinations. The search ranges were: learning rate (0.005 to 0.05, step 0.005), ${\lambda }_2$ (0.005 to 0.05, step 0.005), $L$ (1 to 6, step 1), and $H$ (10 to 50, step 10). Early stopping identified optimal epochs. Other hyperparameters were fixed: $d_k=53,m=26$, and ${\lambda }_1=1$. The optimal parameter combination was defined as the one that achieved the lowest ${ℒ}_{\mathit{\text{pre}}}$ loss. We then retrained the model using the optimal parameters derived from the outer training data. The model with the lowest loss across the five models in the outer loop was selected for subsequent fine-tuning tasks. The hyperparameters determined by grid search are: learning rate=0.015, ${\lambda }_2=0.005,L=4,H=20$, and epoch=274.

4.2. Classification fine-tuning setup

4.2.1. Data acquisition and FC calculation

We evaluated the model on two classification tasks: gender and mental disorder classification. For gender classification that classifies females and males, we utilized fMRI data from the following datasets: (1) UK Biobank with 6,694 subjects (Female: 3,349, Male: 3,345), which was the same dataset used in unsupervised pretraining but for a different purpose here, ensuring unbiasedness; (2) human connectome project (HCP) with 1,092 subjects (Female: 594, Male: 498). The demographics are provided in Table S2 (Supplementary Materials).

For mental disorder classification, we distinguished patients from healthy controls (HCs) across multiple datasets. For schizophrenia (SZ), we combined data from the bipolar and schizophrenia network on intermediate phenotypes (BSNIP), the center for biomedical research excellence (COBRE), the functional biomedical informatics research network (FBIRN), and the Maryland psychiatric research center (MPRC), totaling 1,022 subjects (567 HCs, 455 SZ). For autism spectrum disorder (ASD), we used autism brain imaging data exchange (ABIDE) I and II datasets with 1,610 subjects (881 HCs, 729 ASD).

FC matrices for all datasets were computed identically to the UK Biobank procedure. The demographic details of the data are provided in Table S2 of the Supplementary Materials.

4.2.2. Implementation details

We applied nested 5-fold cross-validation as described in pretraining. Inner loops used grid search over hyperparameters: learning rate (0.0005 to 0.005, step 0.0005), ${\lambda }_1$ (0.1 to 1, step 0.1), and ${\lambda }_2$ (0.005 to 0.05, step 0.005). Early stopping determined optimal epochs. Other parameters were fixed: $d_k=53,m=26,{\lambda }_3=1,L=4$, and $H=20$. Performance was measured on a 5-fold external test set using accuracy, precision, recall, and the area under the receiver operating characteristic curve (AUC). Means and standard deviations were reported. Detailed information on the hyperparameters chosen for the grid search is provided in Table S3 of the Supplementary Materials.

4.2.3. Results of the classification task

We compared our method to classical and recent models: SVM, MLP, GCN [32], BrainGNN [33], BrainNetTF [17], GT [16], GraphTrans [13], and SGFormer [14]. All models used the same data splits and evaluation metrics. Hyperparameters were selected via grid search with early stopping to avoid overfitting, ensuring fair comparison.

Table 1 summarizes performance across tasks and models. For UK Biobank gender classification, our method achieved 85.83% accuracy, 85.99% precision, 85.82% recall, and 93.89% AUC, outperforming SGFormer by 3.75% accuracy and 3.67% AUC. On HCP data, accuracy was 84.08%, precision 84.00%, recall 84.07%, and AUC 91.68%, surpassing SGFormer by 4.15% accuracy and 4.07% AUC. These results highlight the robustness of our model in identifying gender-related brain features, with superior performance compared to several state-of-the-art methods, including BrainGNN, BrainNetTF, GT, GraphTrans, and SGFormer.

For mental disorders, our method reached 75.34% accuracy, 75.41% precision, 75.24% recall, and 82.16% AUC for SZ, beating SGFormer by 2.93% accuracy and 2.85% AUC. For ASD, accuracy was 67.33%, precision 67.13%, recall 67.13%, and AUC 72.40%, exceeding SGFormer by 3.91% accuracy and 4.92% AUC. This demonstrates the model’s effectiveness in distinguishing between HCs and individuals with SZ or ASD.

4.3. Regression fine-tuning setup

4.3.1. Data acquisition and FC calculation

We evaluated age prediction using two datasets: UK Biobank (6,694 subjects, ages 44–80) and HCP (1,092 subjects, ages 22–37). FC matrices for HCP were computed using the same method as UK Biobank. Demographics are in Table S2 (Supplementary Materials).

4.3.2. Implementation details

A nested 5-fold cross-validation framework, consistent with classification fine-tuning, was used. Hyperparameters were tuned by grid search within the same ranges as classification. Early stopping determined the optimal epoch number. Performance was assessed on an external 5-fold test set using mean absolute error (MAE, years) and Pearson correlation (Corr). Mean and standard deviation over folds were reported.

4.3.3. Results of the regression task

We compared our model to SVR, MLP, GCN [32], BrainGNN [33], BrainNetTF [17], GT [16], GraphTrans [13], and SGFormer [14]. All models used the same data splits and evaluation protocol. Hyperparameters were selected via grid search with early stopping to avoid overfitting, ensuring fair comparison.

Table 2 summarizes brain age prediction results. On UK Biobank, our method achieved MAE 5.8447 and Corr 0.4220, outperforming SGFormer by 0.1641 MAE and SVR by 0.1131 Corr. On HCP, MAE was 2.9655 and Corr 0.2567, surpassing GraphTrans by 0.1836 MAE and SGFormer by 0.0474 Corr. Our model consistently outperformed all competitors across both datasets. These results suggest that our fine-tuning framework is not only effective for classification tasks but also adaptable to prediction tasks.

Table 2.

Four quantitative metrics outcomes from the comparative experiments predicting gender and mental disorder classification

Task	Dataset	Method	Accuracy (%)	Precision (%)	Recall (%)	AUC (%)
Gender classification (Female vs. Male)	UK Biobank	SVM	77.90±1.43	78.00±1.40	77.91±1.43	85.70±1.76
		MLP	76.70±1.31	77.02±1.05	76.70±1.30	85.11±0.76
		GCN	80.28±1.08	80.63±1.41	80.28±1.08	88.83±0.95
		BrainGNN	80.45±0.97	80.94±1.13	80.44±0.97	89.10±0.91
		BrainNetTF	80.33±0.90	80.92±0.50	80.32±0.90	88.89±0.99
		GT	81.22±1.03	81.34±1.11	81.22±1.03	89.34±0.74
		GraphTrans	79.12±1.01	79.82±2.39	78.30±5.81	87.73±1.02
		SGFormer	82.08±1.13	82.28±0.93	82.07±1.13	90.22±0.65
		Ours	85.83±1.19	85.99±1.17	85.82±1.19	93.89±0.81
Gender classification (Female vs. Male)	HCP	SVM	77.03±5.90	77.02±6.03	76.64±6.20	82.71±6.77
		MLP	77.22±4.34	77.59±3.07	69.08±8.63	85.66±4.71
		GCN	73.96±2.32	74.06±2.31	74.01±2.50	81.33±3.33
		BrainGNN	74.87±2.51	74.94±2.66	74.36±2.13	82.38±3.02
		BrainNetTF	73.23±3.43	73.15±3.34	72.79±3.41	79.01±3.30
		GT	76.13±4.25	75.97±4.31	75.86±4.12	84.27±3.83
		GraphTrans	73.41±4.41	74.58±4.48	72.65±5.36	84.31±3.58
		SGFormer	79.93±3.58	80.86±4.00	79.69±6.13	87.61±3.65
		Ours	84.08±3.01	84.00±3.09	84.07±3.13	91.68±2.67
Mental disorder classification (HC vs. SZ)	BSNIP COBRE FBIRN MPRC	SVM	67.22±2.21	67.34±2.33	67.51±2.36	73.76±2.45
		MLP	71.24±5.27	68.77±6.37	64.83±6.58	76.75±6.20
		GCN	73.87±3.15	74.61±2.53	73.05±3.74	79.26±2.68
		BrainGNN	71.52±2.46	71.43±2.63	71.43±2.88	77.30±3.93
		BrainNetTF	69.18±2.60	69.02±2.70	68.59±2.60	74.26±3.50
		GT	71.52±3.14	71.46±3.30	71.36±3.49	78.17±4.18
		GraphTrans	67.02±4.44	67.27±4.71	67.05±4.61	73.00±5.14
		SGFormer	72.41±3.10	72.72±3.73	72.22±3.24	79.31±5.86
		Ours	75.34±3.32	75.41±3.39	75.24±3.47	82.16±3.48
Mental disorder classification (HC vs. ASD)	ABIDE I ABIDE II	SVM	62.36±2.79	64.02±2.06	63.16±2.32	70.29±1.72
		MLP	61.43±2.00	63.90±3.06	58.63±2.16	68.11±3.71
		GCN	62.11±2.20	64.84±1.60	62.94±1.78	69.67±2.19
		BrainGNN	62.36±2.79	63.97±1.99	63.14±2.30	70.30±1.74
		BrainNetTF	60.81±2.48	60.52±2.35	60.22±2.61	65.29±2.86
		GT	63.04±2.93	63.83±2.43	63.54±2.46	68.03±2.42
		GraphTrans	61.18±3.95	61.33±3.66	60.37±3.70	65.21±4.38
		SGFormer	63.42±1.77	63.21±1.70	62.88±2.31	67.48±2.48
		Ours	67.33±1.53	67.13±1.44	67.13±1.34	72.40±2.10

Note: The results are presented as mean ± standard deviation, calculated from the outer loop of a nested 5-fold cross-validation.

4.4. Clustering fine-tuning setup

4.4.1. Data acquisition and FC calculation

We performed biotype identification for unspecified depressive and anxiety disorders. Biotypes refer to subgroups with distinct FC patterns. The data from the UK Biobank included 88 subjects; after excluding 15 with comorbidities, 36 remained for depression and 37 for anxiety. FC matrices were computed using the same method as pretraining. Demographic details are in Table S2 (Supplementary Materials).

4.4.2. Implementation details

Unlike classification, clustering used the full dataset without train-test splits. Hyperparameters were set as: learning rate=0.001, $d_k=53,m=26,{\lambda }_1=0.5,{\lambda }_2=0.005,{\lambda }_3=1,L=4$, and $H=20$. Model performance was assessed using external metrics such as accuracy (ACC), F1 score, normalized mutual information (NMI), and adjusted rand index (ARI) to measure agreement with clinical labels. Internal metrics, including silhouette coefficient index (SCI) and Davies-Bouldin index (DBI), were used to evaluate cluster quality.

4.4.3. Results of the clustering task

We compared our model with K-means, spectral clustering (SC), DEC [34], GAE [20], GALA [22], and DGAE [24]. The evaluation methods were consistent across all methods.

Table 3 shows our model outperformed the second-best DGAE by 0.0137 in ACC, 0.0123 in F1, 0.0172 in NMI, 0.0228 in ARI, and 0.0108 in SCI, while reducing DBI by 0.0243. These results demonstrate superior identification of depressive and anxiety biotypes and overall clustering quality, highlighting our model’s robustness and adaptability.

Table 3.

The two quantitative metrics outcomes for the comparative experiments predicting brain age

Task	Dataset	Method	MAE	Corr
Brain age prediction	UK Biobank	SVR	6.2406±0.4056	0.3089±0.0379
		MLP	6.1182±0.0447	0.2153±0.0255
		GCN	6.0899±0.0719	0.2640±0.0338
		BrainGNN	6.1400±0.1187	0.2561±0.0323
		BrainNetTF	6.3637±0.0835	0.2562±0.0464
		GT	6.3360±0.0659	0.3018±0.0477
		GraphTrans	6.3023±0.0161	0.2940±0.0644
		SGFormer	6.0088±0.0883	0.2675±0.0528
		Ours	5.8447±0.0661	0.4220±0.0174
Brain age prediction	HCP	SVR	3.2651±0.2752	0.1797±0.0774
		MLP	3.3885±0.0605	0.0212±0.0552
		GCN	3.2811±0.2595	0.1618±0.0440
		BrainGNN	3.2013±0.1285	0.2057±0.0626
		BrainNetTF	3.3165±0.2494	0.1243±0.0416
		GT	3.1873±0.0681	0.1171±0.1112
		GraphTrans	3.1491±0.0114	0.1510±0.1147
		SGFormer	3.1801±0.0281	0.2093±0.1137
		Ours	2.9655±0.1195	0.2567±0.1310

Note: The results are presented as mean ± standard deviation, calculated from the outer loop of a nested 5-fold cross-validation.

5. Discussion

5.1. Ablation studies

We conducted ablation experiments to evaluate key components’ contributions. We compared the full model with versions lacking pretraining, edge features, or the edge-node decoder block. For classification and regression, we also tested the effect of the task token. Detailed quantitative results are in the Supplementary Materials.

Across classification, regression, and clustering tasks, removing pretraining, edge feature encoding, or the edge-node decoder consistently degraded performance, confirming these components’ importance. Specifically, excluding pretraining, edge features, the edge-node decoder block, or the task token lowered classification metrics (Fig. S1). For regression, their removal increased MAE and decreased correlation (Fig. S2). In clustering, excluding pretraining, edge features or the edge-node decoder reduced ACC, F1, NMI, ARI, and SCI (Fig. S3). These findings confirm that all components are essential for optimal performance.

5.2. Test-retest reliability evaluation

We assessed the pretrained model’s test-retest reliability using data from 30 healthy adults in the CoRR-HNU dataset [35]. Each participant underwent ten fMRI scans, conducted every three days over one month. Data preprocessing followed the same steps as those used during pretraining to ensure consistency and accuracy.

Stability was assessed using the intraclass correlation coefficient (ICC) across 10 repeated scans. For each sample, the ICC was calculated by gathering ratings for each feature across the scans, treating each scan as a different “rater” of the same “subject.” The mean ICC for each feature was computed, and the overall stability of the scans was measured by averaging the ICCs across all samples. Table 4 presents the ICC values for features $A,A^{\prime }$, and $\hat{A}$, which were 0.6080±0.0473, 0.6262±0.0456, and 0.6125±0.0474, reflecting moderate reliability. Features $X,X^{\prime }$, and $\hat{X}$ exhibited higher ICC values of 0.7777±0.0530, 0.7966±0.0495, and 0.7942±0.0516, indicating good consistency. These results confirm stable feature representations.

Table 4.

The six quantitative metrics outcomes from the comparative experiments performing depression and anxiety identification

Task	Dataset	Method	ACC	F1 score	NMI	ARI	SCI	DBI
Biotype identification (depression vs. anxiety)	UK Biobank	K-means	0.6027	0.5705	0.0564	0.0443	0.0717	3.3476
		SC	0.5479	0.5307	0.0082	0.0016	0.0663	3.3542
		DEC	0.5753	0.5136	0.0189	0.0142	0.3247	0.8947
		GAE	0.6164	0.6146	0.0209	0.0098	0.3306	1.2611
		GALA	0.6712	0.6717	0.0306	0.0288	0.4842	0.8679
		DGAE	0.6986	0.6992	0.1163	0.1461	0.5163	0.8011
		Ours	0.7123	0.7115	0.1335	0.1689	0.5271	0.7768

Fig. 2 shows t-SNE projections of the pretrained model’s original, embedding, and reconstructed features, visualizing their distributions in two dimensions. Fig. 2(a), Fig. 2(b), and Fig. 2(c) depict $A,A^{\prime }$, and $\hat{A}$; Fig. 2(d), Fig. 2(e), and Fig. 2(f) show $X,X^{\prime }$, and $\hat{X}$. The clusters appear well-separated and demonstrate the effectiveness of our method in capturing intra-subject similarity and inter-subject variability. The quantitative reliability results in Table 4 and the qualitative visualizations in Fig. 2 confirm the method’s effectiveness in preserving reliable and discriminative representations across repeated measurements.

Fig. 2.

t-SNE visualization results for the CoRR-HNU dataset. (a) t-SNE visualization of $A$. (b) t-SNE visualization of $A^{\prime }$. (c) t-SNE visualization of $\hat{A}$. (d) t-SNE visualization of $X$. (e) t-SNE visualization of $X^{\prime }$. (f) t-SNE visualization of $\hat{X}$.

5.3. Identified discriminative brain regions

Our model interprets task-specific brain region importance via attention weights from node features and adjacency embeddings. After introducing a task-specific token node, the contribution of each brain region is identified by the corresponding position in the attention matrix related to that token (Fig. 1(d)). For each task, we selected the five brain regions that contributed the most (Fig. 3). These regions align with existing studies, helping the model identify areas linked to neuropsychiatric conditions and cognitive functions, thus aiding understanding of underlying neuropathology.

Fig. 3.

Key brain regions identified across various tasks including gender classification, mental disorder classification, brain age prediction, and biotype identification. (a) Gender classification for the UK Biobank and HCP datasets. (b) Mental disorder classification for the four SZ datasets. (c) Mental disorder classification for the two ASD datasets. (d) Brain age prediction for the UK Biobank and HCP datasets. (e) Biotype identification for depressive disorder. (f) Biotype identification for anxiety disorder.

The identified regions match findings from multiple studies (details in Supplementary Materials). Gender classification highlighted the AU and SM domains, reflecting gender-related differences in auditory processing and motor control (Fig. 3(a)). Mental disorder classification showed distinct patterns: SZ involved SM and AU disruptions related to motor and auditory deficits (Fig. 3(b)) [36]; ASD involved DM and SM domains linked to self-referential processing and sensorimotor integration (Fig. 3(c)) [37]. Brain age prediction emphasized SM and CC domains, associated with motor decline and executive dysfunction (Fig. 3(d)) [38]. In clustering, key regions distinguished anxiety and depression. Fig. 3(e) and Fig. 3(f) show that depression involves the CC domain, linked to cognitive deficits, while anxiety relates to the DM domain, associated with altered self-referential processing [39, 40]. This interpretability validates the model’s biological basis and offers clinicians a tool to identify biomarkers for early diagnosis and personalized treatment, enhancing neuroimaging’s application value.

5.4. Identified discriminative FCs

The attention weights indicate the strength of FC between brain regions during tasks. To assess FC, we analyzed the top 20% of strongest connections for each of the five key brain regions identified earlier. For each region, connections with weights within this upper range were considered its most significant links. This analysis reveals complex network changes related to diverse neurological and behavioral traits.

In gender classification (Fig. 4(a)), most key FCs involved regions in the VI domain, indicating concentrated connectivity. Mental disorder analysis showed that SZ primarily involved SM, VI, and AU domains (Fig. 4(b)) [36]. ASD involved SM, VI, CC, and DM domains, reflecting diverse disruptions (Fig. 4(c)) [37]. Brain age prediction (Fig. 4(d)) highlighted CC, CB, and SM domains, with CC dominant. For clustering, depression involved four domains (Fig. 4(e)) with CC contributing most, followed by DM and SM [39]. Anxiety involved five domains (Fig. 4(f)), with CC leading and the CB domain also present [40]. These FC patterns confirm known neuropathological changes and show our model’s strength in detecting subtle, biologically relevant network biomarkers.

Fig. 4.

Key FCs identified across various tasks including gender classification, mental disorder classification, brain age prediction, and biotype identification. (a) Gender classification for the UK Biobank and HCP datasets. (b) Mental disorder classification for the four SZ datasets. (c) Mental disorder classification for the two ASD datasets. (d) Brain age prediction for the UK Biobank and HCP datasets. (e) Biotype identification for depressive disorder. (f) Biotype identification for anxiety disorder. The blue edges represent the identified unidirectional connections (with directions pointing toward the five brain regions selected for each task), while the red edges indicate the identified bidirectional connections.

6. Conclusions and future work

We propose a novel foundation model for FCNs using a graph transformer-based autoencoder to extract node and edge embeddings. The model performs well in reconstruction and adapts effectively to classification, regression, and clustering tasks. It also identifies meaningful biomarkers that aid interpretation of neural mechanisms. Validated on seven neuroimaging datasets, our model shows robustness and broad applicability in neuroscience. This work bridges graph representation learning, foundation model design, and neuroimaging applications.

Despite these contributions, several limitations remain, offering directions for future research. First, the model’s performance accuracy, while promising, could be affected by potential label noise in the datasets. Future work should explore methods to improve model robustness under noisy labels. Second, the model currently requires considerable effort for task transfer. Future research could focus on simplifying the transfer process across similar tasks to enhance the model’s practicality and applicability.

Table 5.

Test-retest ICC results

Feature	ICC
Original feature $A$	0.6080±0.0473
Embedding feature $A\prime$	0.6262±0.0456
Reconstructed feature $\hat{A}$	0.6125±0.0474
Original feature $X$	0.7777±0.0530
Embedding feature $X\prime$	0.7966±0.0495
Reconstructed feature $\hat{X}$	0.7942±0.0516

Data availability statement

The datasets can be accessed via the link in Table S2 of the Supplementary Materials. The code, pretrained model, and NeuroMark toolbox are available at http://www.yuhuidu.com/.