Assisting schizophrenia diagnosis using clinical electroencephalography and interpretable graph neural networks: a real-world and cross-site study

Abstract

Schizophrenia (SCZ) is a chronic and serious mental disorder with a high mortality rate. At present, there is a lack of objective, cost-effective and widely disseminated diagnosis tools to address this mental health crisis globally. Clinical electroencephalogram (EEG) is a noninvasive technique to measure brain activity with high temporal resolution, and accumulating evidence demonstrates that clinical EEG is capable of capturing abnormal SCZ neuropathology. Although EEG-based automated diagnostic tools have obtained impressive performance on individual datasets, the transportability of potential EEG biomarkers in cross-site real-world application is still an open question. To address the challenges of small sample sizes and population heterogeneity, we develop an advanced interpretable deep learning model using multimodal clinical EEG features and demographic information as inputs to graph neural networks, and further propose different transfer learning strategies to adapt to different clinical scenarios. Taking the disease discrimination of health control (HC) and SCZ with 1030 participants as a use case, our model is trained on a small clinical dataset (N = 188, Chinese) and enhanced using a large-scale public dataset (N = 508, American) of adult participants. Cross-site validation from an independent dataset of adult participants (N = 157, Chinese) produced stable performance, with AUCs of 0.793–0.852 and accuracies of 0.786–0.858 for different SCZ prevalence, respectively. In addition, cross-site validation from another dataset of adolescent boys (N = 84, Russian) yielded an AUC of 0.702 and an accuracy of 0.690. Moreover, feature visualization further revealed that the ranking of feature importance varied significantly among different datasets, and that EEG theta and alpha band power appeared to be the most significant and translational biomarkers of SCZ pathology. Overall, our promising results demonstrate the feasibility of SCZ discrimination using EEG biomarkers in multiple clinical settings.

Introduction

SCZ is a chronic and complex neuropsychiatric disorder affecting about 21 million people globally [1]. It is characterized by heterogenous positive (e.g., hallucinations and delusions) and negative (e.g., blunted affect) symptoms and is associated with functional impairments, decreased quality of life, and substantial mortality [2, 3]. In view of the global mental health crisis with unbalanced resources [4], accurate and timely screening/diagnosis of SCZ, especially with objective and feasible assessments in real-world settings, is crucial for precise prevention and intervention [2, 3].

Despite great efforts in developing the artificial intelligence (AI)-aided technologies for SCZ diagnosis [5–9], interpretable models, especially those transportable across different clinical sites, are still lacking [6, 10]. Previously, schizophrenia detection was performed using several widely accepted neuroimaging techniques, such as positron emission tomography (PET), magnetic resonance imaging (MRI), and functional MRI (fMRI) [11]. However, these neuroimaging techniques have high cost of imaging equipment and high demands on operational skills, limiting their wide dissemination in clinical practice. Therefore, there is an urgent need for easy-to-use and cost-effective brain monitoring techniques to aid in the diagnosis of schizophrenia.

This requirement can be fulfilled by electroencephalogram (EEG) technology, which enables the noninvasive monitoring of electrical activity in the brain using scalp electrodes. As an effective indicator of neuropathology, EEG shows great potential in the automatic diagnosis of neurological disorders and mental disorders such as epilepsy [12], Alzheimer’s disease [13], and depression [14]. Despite having a lower spatial resolution compared to other neuroimaging techniques like fMRI, the superior temporal resolution of EEG encompasses a wealth of temporal dynamic information. Recently, increasing evidences have shown that EEG is capable to capture abnormal neuropathology, providing rich clues for the diagnosis of SCZ [5]. For example, SCZ patients were found to exhibit power disruptions across various frequency bands [15]. Considering this premise, numerous studies have employed machine learning (ML) techniques, encompassing both traditional or deep learning (DL) methods, to construct classification models based on EEG features [10, 16–22]. These studies propose that EEG signals contain discriminative information valuable for SCZ diagnosis, and that ML serves as an effective tool to help accomplish this objective. Promising performance was obtained in these studies, with accuracies ranging from 81.5–99% in different datasets (Supplementary Table 1).

However, the translational ability has long been a barrier to widespread dissemination of EEG-based automatic diagnosis tools [10, 12]. The daunting challenges for generalization of mental disorder classifier are population heterogeneity and real-world sampling variability [10, 23]. Furthermore, the limited data annotation in mental health institutions, especially of HCs, makes it infeasible to build automated diagnosis tools independently. Over the past decades, tremendous efforts have been spent on resolving MRI issues of sample sizes, heterogeneity and sampling variability [24], leading to promising results from large multi-center studies with several hundreds of participants [25]. On the contrary, applications of EEG for SCZ diagnosis are still at the early stage. Previous EEG studies have primarily worked on small-scale datasets from a single site which are prone to overfitting and cannot cover the diverse population heterogeneity [26]. Therefore, the accuracies from these studies should be interpreted with caution. Besides, although multiple approaches have been proposed to remove artifacts from EEG signals [27], the EEG datasets used in previous studies were mostly sampled from research lab settings rather than collected from real-world clinical procedures, which may have more noisy signals. Surprisingly, it should be noted that few studies have examined the performance of EEG-based diagnosis tools generated from one population on other populations, making the transportability of potential EEG biomarkers across population heterogeneity an open question.

Furthermore, transfer learning (TL) techniques in deep neural networks (DNNs) effectively transfer knowledge or patterns learned from (large-scale) available data to new tasks. Thus, the performance of new tasks could be improved with reduced data annotation in a more cost-effective way [28–33]. In particular, graph neural networks (GNNs) [34, 35] have demonstrated unique advantages in representing complex brain structures (such as connections between channels in different brain regions) in EEG-based BCI (Brain-Computer Interface) applications [36, 37]. However, few TL studies have worked on improving the generalizability and wide dissemination of EEG-based diagnosis tools for mental disorders. As a proof-of-concept study, taking the differential diagnosis of HCs and SCZs as a use case, we investigated the direct cross-site transportability of EEG-based diagnosis models and the feasibility of employing TL techniques to improve the transportability:

Our study design of classification-based differential diagnosis between HCs and SCZs was illustrated in Fig. 1. First, eligible 1030 participants were collected from four data sources: including 508 HCs from TUH (USA) and other three independent groups from Chengdu (China, 95 HCs/93 SCZs), Hangzhou (China, 56 HCs/194 SCZs), and Moscow (Russia, 39 HCs/45 SCZs), respectively. Next, a range of interpretable frequency/time-domain EEG features were extracted, and graph convolutional neural networks (GCNs) were used to build the diagnosis models. Semantically interpretable multichannel EEG biomarkers and demographic information were used as features. Furthermore, different TL strategies were investigated in an integrated framework, EEG_TL, to increase the diagnosis classifier transportability. The performance of direct cross-site evaluation and employing TL techniques was investigated systematically, with comparisons to different structures of EEG graphs and other classical DL/ML methods. Feature importance related to SCZ pathology was also measured, visualized, and validated based on neuropathology analysis. In particular, salient EEG biomarkers with strong transportability across multiple sites were explored.

Fig. 1. Study design of classification-based diagnosis of schizophrenia.

a The diagnosis models were trained on the Chengdu dataset, with augmentation from the TUH dataset. The Hangzhou and Moscow datasets were used for cross-site validation of the trained models; b Graph convolutional neural networks (GCNs) were used to build the diagnosis models based on multimodal EEG biomarkers and demographic features; c multiple transfer learning strategies were integrated as a unified framework, EEG_TL, to increase the translational ability of the diagnosis models in flexible clinical settings.

Methods

Participants

In total, four EEG datasets were used in this study: the TUH dataset only contained HCs and was used as an auxiliary resource for transfer learning; the Chengdu dataset was used to build automatic diagnosis models and the rest three, namely Hangzhou, Moscow were used for cross-site performance validation. All samples were collected in the eyes-closed resting state of participants using the standard 10–20 EEG montage. For simplicity, each dataset is referred to by its original location in this study. Details of each dataset are as follows:

TUH

508 HCs were filtered out from more than 1400 subjects with normal neurological pathology in the open-accessed TUH Abnormal EEG Corpus [38]. According to the physician’s report for each EEG recording, subjects with mental disorders, brain injuries, and epilepsy were excluded. Subjects with age between 20 and 45 years old were selected. The original EEG recordings had durations ranging from 15–48 min, sampled with 33 channels at 250 HZ.

Chengdu

we recruited 93 inpatients from West China Hospital (Chengdu, China), diagnosed with SCZ based on ICD-10 criteria and 95 HCs through advertisement between January 2014 and February 2021. We included only patients who had no history of other mental disorders, and HCs were ensured that they had no personal or immediate family history of mental illness. Exclusion criteria include (1) current or past history of neurological or neurodevelopmental disorders, substance-related problems, or head trauma with loss of consciousness, (2) left-handedness, and (3) any other EEG contraindications. All patients aged between 18 and 65 collected 3 minutes of clinical EEG data at the beginning of their routine examination. The EEG data was collected by 16-channel Shanghai Nuocheng device at a sampling frequency of 128 Hz. The study was approved by the local ethic committee.

Hangzhou

We recruited 194 SCZ inpatients from Hangzhou Seventh Hospital and 56 HCs between November 2021 and April 2022 in Hangzhou, China. The protocol in Hangzhou was similar to Chengdu. Unlike Chengdu, the EEG data was acquired by 19-channel Beijing Solar device, and the sampling frequency was 256 Hz.

Moscow

We used the open-access EEG database, which was obtained and prepared by Dr. Gorbachevskaya (leading researcher at the Mental Health Research Center) and Dr. S.V. Borisov (senior researcher at the faculty of Biology M.V. Lomonosov Moscow State University) in the Laboratory for Neurophysiology and Neuro-Computer Interfaces at M.V. Lomonosov Moscow State University [39]. The EEG signal was recorded from two groups of male adolescent individuals, including 39 HCs and 45 with SCZ. Each EEG signal of this dataset consists of 16 channels. These recorded EEG signals are of 1-min duration, each containing 7680 EEG voltages of the order of mV, and are sampled at 128 Hz.

Data preprocessing

We used Automagic [40], an open-source MATLAB toolbox, to standardize preprocessing of big EEG data. Automagic has proven to be an effective method to reduce a large extent of EEG by applying a pipeline of algorithms to identify artifactual channels in combination with multiple artifact rejection algorithms. All EEG data from the four different sites were preprocessed using the same procedure. The standardized preprocessing pipeline includes the following steps: (1) 0.1–40 Hz bandpass filtering; (2) removal of flatline channels, low-frequency drifts, low-correlation channels and noise channels; (3) application of Multiple Artifact Rejection Algorithm (MARA) for artifact correction; (4) channel interpolation and referencing.

To assure the data quality, we took the EEG recordings between the 3rd minute to the 15th minute of TUH EEG corpus as the samples to avoid the interference caused by signals of electromyography and electrooculogram at the beginning of the subject’s EEG acquisition and to avoid the fatigue and sleepiness of the subject during the longer acquisition. To be consistent with Chengdu and Hangzhou, the recordings were split into 3232 samples of 2-minute long, with a 1-minute overlap between adjacent samples.

Besides, to address the long-tailed distribution of the time-domain characteristics of TUH EEG corpus, we introduce the Box–Cox transformation [11], which makes the distribution of TUH EEG corpus more similar to the normal distribution. The Box–Cox transformation equation is as follows.

$$y\left(\lambda \right)=f\left(x\right)=\left\{\begin{array}{c}\hfill \frac{y_i^{\lambda }-1}{\lambda },\lambda \ne 0\hfill \\ \hfill ln\left(y_i\right),\lambda =0\hfill \end{array}\right)$$ 1

Feature generation

Sixteen channels common in all four datasets are selected, including FP1, FP2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T3, T4, T5, and T6. The electrode distribution for all channels is set according to the international 10-20 system. When considering features of EEG signal representation, although original time series are often used in DL-based methods, they are more prone to overfitting and have poor generalizability in external evaluations [36]. To simplify the EEG signal representations for a better generalizability, this study extracted semantically interpretable frequency/time-domain biomarkers [41, 42] that have already been validated in traditional ML methods, and applied them in DL-based models for the first time. As listed in Supplementary Table 2, we have extracted two types of frequency-domain features and four types of time-domain features to represent each channel of EEG signals. In addition, demographic features of each subject are also considered because of the large individual discrepancy in EEG signals.

1. Frequency-domain EEG feature: absolute power and differential entropy (DE).

2. Time-domain EEG feature: energy, amplitude, mean value, variance, 1st order difference, and 2nd order difference of signals, 1st order difference and 2nd order difference of normalized signals, Hjorth activity, Hjorth mobility, and Hjorth complexity, Petrosian fractal dimension.

3. Demographic feature: age and gender.

In particular, we obtain frequency features from five different rhythms, including delta (1–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (30–44 Hz). Thus, we can obtain 10 different frequency features for each channel. However, different sites employ various EEG devices, introducing measurement variability between the datasets. To address this issue, we implement a min-max normalization technique for all extracted features in each dataset independently prior to inputting them into the ML or DL models. This normalization step helps mitigate the effects of confounding factors, such as impedance difference, and ensures a fair comparison across the datasets. The two types of demographic characteristics are the age and gender of each subject.

EEG-based diagnosis model using transfer learning

Transfer learning strategies

Three TL strategies were employed in this study to improve the transportability of EEG-based diagnosis models: large-scale of HC recordings in the external TUH dataset were leveraged for pre-training EEG graph representations (PT) and meta learning-based data augmentation (META). For PT (Supplementary Fig. 1), EEG graph representations learned from TUH were fine-tuned on the Chengdu data for diagnosis classifier training. For META (Supplementary Fig. 2), TUH recordings were added into the training data as an augmentation for model learning. Both PT and META aimed to enrich the EEG representations for a better generalizability. In addition, unsupervised domain adaptation (DA) was implemented during cross-site classification. For DA, unlabeled samples from the target site were used to reduce the discrepancies between the source data (where the model is originally trained) and the target data, so that to adapt the classifier to the target site. More details about the TL strategies are elaborated in the method introduction in supplementary materials.

Diagnosis model building

EEG graph construction

A left-right symmetric graph structure was constructed to represent EEG signals of 16 channels in GCN, as shown in Fig. 2a. Multimodal (frequency/time) EEG features were extracted to represent each of the sixteen channels, namely each node in the graph. Edges between node pairs were created to represent connections between channels in the graph. EEG features and demographic features were first represented independently and then fused together for classification.

Fig. 2. Graph structure of EEG channels used in the graph convolutional neural networks.

EEG signals are collected from 16 channels, including FP1, FP2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T3, T4, T5, and T6. Each channel corresponds to a node in the graph, and edges are used to connect symmetric node pairs of FP1 and FP2, F3 and F4, C3 and C4, P3 and P4, and O1 and O2. Three graph structures in (a)–(c) are compared in terms of transferable ability of their pre-trained EEG graph models. The one in (a) is selected because it outperforms in downstream five-fold cross validation on Chengdu.

Diagnosis model training

The diagnosis model was primarily trained using the Chengdu data. The GCN model pre-trained on the TUH dataset can be employed to initialize the parameters of the diagnosis classifier for fine-tuning. Additionally, the healthy control (HC) data from the TUH dataset are utilized as an augmentation to enrich the HC patterns in the training data. Specifically, we designed two meta-learners, which include a schizophrenia (SCZ) classifier based on contrastive learning and a domain discriminator to align the feature distributions across different datasets. In each iteration (i.e., every 300 epochs), HCs from the TUH dataset are randomly sampled in equivalent size and integrated into the training data as an augmentation. These two meta-learners are then used to incrementally optimize performance.

The SCZ classifier comprises a fully-connected layer with 2 output units, connected to a Softmax activation function. The InfoNCE loss function of contrastive learning is employed to enhance the classification performance (see Supplementary Equation 19). This approach aims to minimize the distance between a given sample (i.e., a query) and samples of the same class (i.e., positive samples) while maximizing the distance between a given sample and samples from another class (i.e., negative samples).

Similarly, the domain discriminator also consists of a fully-connected layer of 2 output units, connected with a Softmax activation function. To illustrate, when aligning the TUH data (domain s) and the Chengdu data (domain t) as an example, let $F^s=\left\{f_1^s,f_2^s,\cdots ,\right)\left(f_n^s\right)$ and $F^t=\left\{f_1^t,f_2^t,\cdots ,\right)\left(f_n^t\right)$ denote their respective features. During training, domain-invariant features are rewarded to reduce variance of the marginal distribution of the source and target data (see Supplementary Equation 20).

The diagnosis classification model is trained on the GCN model of EEG graph representation using the following loss function:

$$\mathcal{L}\left({\theta }_f,{\theta }_c,{\theta }_d\right)=\alpha {\mathcal{L}}_c+\beta {\mathcal{L}}_d$$ 2

where α, β are the tradeoff parameters to balance the SCZ classifier ${\mathcal{L}}_c$ and the domain discriminator ${\mathcal{L}}_d$ for optimal performance.

More details of the diagnosis model training can be found in Supplementary eMethods.

Cross-site diagnosis

In cross-site diagnosis, an unlabeled sample from the external site went through the diagnosis model to identify its label as HC or SCZ. In this stage, the TL strategy of DA could be implemented, by sending the unlabeled sample through the domain discriminator as defined above to reduce the discrepancy between data from different sites. In this way, the diagnosis classifier was adapted to the external site and the diagnosis performance was boosted (Supplementary Fig. 3).

Evaluation

Experimental setup

(1) Firstly, an intra-site evaluation was conducted on the Chengdu dataset using five-fold cross validation. (2) Next, a cross-site evaluation was conducted on the Hangzhou and Moscow datasets using classification models trained from the entire Chengdu dataset. In particular, subsets of different HC/SCZ splits from the Hangzhou dataset were used to test performance stability with diverse SCZ prevalence and population heterogeneity, as suggested by Fernandes et al. [7]. In addition, the performance on the Moscow dataset was also examined for the transportability from models trained on adult data to adolescent data. (3) Furthermore, models with/out TL strategies were compared to check the original cross-site performance and potential transportability improvements after using TL strategies. (4) DL algorithms (EEGNet [43], LSTM [44], and GAT [45]) and ML algorithms (SVM [46], Logistic regression [47], Random Forest [48], and XGBoost [49]) commonly used for EEG-based classification were also implemented for performance comparison. Of note, raw EEG signals with time sequences and demographic features were used as features for EEGNet/LSTM, while GAT and all ML algorithms used the same set of features as our proposed GCN. Detailed experiment configuration and (hyper-)parameters were reported in the Supp - Training and parameters section.

Evaluation criteria

Taking SCZ as the positive class, standard metrics, i.e., AUC, ACC, sensitivity, specificity, precision, and F1, were reported to provide an objective model assessment from different aspects. Balanced sensitivity and specificity scores were reported based on the ROC (receiver operating characteristic) curves. All experiments were repeated 50 times, and the mean (±SD) of each evaluation criterion was reported.

Statistical analysis

Statistical analyses were conducted by IBM SPSS Statistics 22 (IBM SPSS, Turkey). A comparison of categorical variables between the two groups was performed using the χ² test. Comparison of continuous variables between two groups were performed using a two-tailed two-sample t test (for examined normal distribution of data). Unless specified otherwise, the significance of all tests was set to p < 0.05. All statistical analyses were done using R (version 4.1).

Ethical aspects

The study was carried out in accordance with ethical principles for medical research involving humans (WMA, Declaration of Helsinki). All data were collected anonymously. The study protocol was approved by the Ethical Committee of the West China Hospital and the Hangzhou Seventh People’s Hospital.

Results

Data statistics

All EEG samples in this study were collected in an eyes-closed resting state using the standard 10–20 EEG montage. In total, four EEG datasets were used in this study, and their demographic statistics is listed in Table 1. Detailed data eligibility and sampling configurations can be found in the “Method” section. More data descriptions are referenced to Supplementary Tables 3 and 4.

Table 1.

Demographics of samples in this study.

	TUH		Chengdu				Moscow
Characteristic	HC (508)		HC (95)		SCZ (93)		HC (39)		SCZ (45)
Age^ (years)	32.54 ± 7.77		25.17 ± 5.22		29.25 ± 10.84**		10–19		10–19
Female#	345 (0.680)		52 (0.547)		46 (0.495)*		0 (0.0)		0 (0.0)
	Hangzhou
Characteristic	HC (56)	SCZ (194) 1:3		SCZ (112) 1:2		SCZ (56) 1:1		SCZ (28) 2:1	SCZ (19) 3:1
Age^ (years)	31.21 ± 12.25	32.27 ± 10.03*		32.04 ± 11.08*		31.19 ± 10.77**		31.40 ± 11.56**	32.14 ± 11.66**
Female#	38 (0.679)	119 (0.613)**		107 (0.637)**		72 (0.643)**		55 (0.655)**	49 (0.662)**

^{^}Values are shown as mean and standard deviation. ^#Values are shown as count and percent of the distribution.

1:3, 1:2, 1:1, 2:1, and 3:1 stand for the ratio between healthy controls (HCs) and schizophrenia (SCZ) patients.

^* Statistically significant difference (p value < 0.05) between HCs and SCZ patients.

^** Statistically significant difference (p value < 0.001) between HCs and SCZ patients.

EEG samples

Original EEG recordings were of different lengths (TUH: 15–45 min, Chengdu/Hangzhou: 3 min, Moscow: 1 min), and they went through a series of automated data cleaning, transformation and splitting before being used in experiments. Detailed information about data processing can be found in the Method section. In a pilot study (Supp - EEG Sampling), EEG samples in the Chengdu and TUH datasets were tested of different lengths (e.g., 1 second, 1 min, 2 min) and of different overlap windows (to generate samples of different scales), based on their five-fold cross-validation performance on the Chengdu data. In the end, a 2-min sample was generated for each participant in the Chengdu and Hangzhou datasets, respectively. A total of 3232 2-min TUH samples were used in the experiment. In contrast, the averaged samples of Moscow only had 56 seconds after data preprocessing.

Intra-site validation

Supplementary Fig. 4 provides the five-fold cross-validation performance on the Chengdu dataset. Supplementary Fig. 4a depicts the result histogram accompanied by a performance table for different method comparison. Performance of the base GCN model, GCN with EEG_TL implementation (PT + META), and other compared DL/ML methods is reported. Supplementary Fig. 4b depicts the ROC curves of the base GCN model, after adding TL strategies of PT, META and PT + META into GCN, respectively. The base GCN model obtained a promising result of AUC/ACC—0.903/0.903. With the implementation of TL strategies, the performance of GCN was improved continuously to 0.932/0.932. EEGNet used raw time series of EEG signals as features and yielded the highest performance (0.968/0.968). Detailed performance of mean (std) values is provided in Supp1ementary Table 5.

Cross-site evaluation

Of note, achieving the optimal performance on a single dataset is not of primary importance in this study. Rather, transportability and performance stability across different sites with various population heterogeneity are more critical pursues here. Cross-site performance on the Hangzhou dataset is provided in Fig. 3. Performance of both direct cross-site evaluation using the base GCN model and EEG_TL integration is provided. Figure 3a depicts the result histogram accompanied by a performance table for different method comparison; Fig. 3b shows a performance comparison among HC/SCZ ratios of 1:3, 1:2, 1:1, 2:1, and 3:1; Fig. 3c illustrates the ROC curves of the base GCN model, after adding each TL strategy and their combinations into GCN, respectively. A HC/SCZ ratio of 1:1 is used in Fig. 3a, c. More detailed performance with different HC/SCZ ratios is illustrated in Supplementary Tables 6–10 and Supplementary Fig. 5.

Fig. 3. Performance comparison for cross-site validation on the Hangzhou dataset.

The ratio between healthy controls (HCs) and schizophrenia patients (SCZs) is 1:1 in (a, c), and the performance of HC/SCZ ratios of 1:3, 1:2, 1:1, 2:1 and 3:1 is shown in (b). The proposed GCN architecture has the strongest translational ability among all the implemented algorithms under different SCZ prevalence. The top 2 best performance in each merit is bolded, and a statistically significant improvement in ACC of EEG_TL compared to other ACCs is noted in (a). Statistically significant difference: ^#P value < 0.05; ^## P value < 0.001. Steady performance improvements can be observed by incrementally implementing different TL strategies. GCN graphic convolutional network, LR logistic regression, EEGNet a compact convolutional neural network, PT pre-trained language model using TUH, META meta learning-based data augmentation using TUH, DA unsupervised domain adaptation using unlabeled data from the test domain.

Performance stability of different DL/ML methods: As shown in Fig. 3a, different EEG representations and DL/ML methods have dramatic differences in cross-site diagnosis performance. The cross-site performance of EEGNet decreased significantly to 0.656/0.656 (vs. 0.968/0.968 on Chengdu), indicating an overfitting and a poor generalizability of the raw EEG time series. Despite well-known transportability and robustness in previous works, the performance of LR also dropped sharply to 0.782/0.732 (vs. 0.921/0.875). In comparison, our base GCN model of multichannel-based graphs with interpretable biomarkers had a relatively small decrease - 0.798/0.798 (vs. 0.903/0.903).

Performance stability across different prevalence: Similar trends as in the HC/SCZ ratio of 1:1 (Fig. 3a) can also be observed in the performance of other ratios (Fig. 3b, Supplementary Tables 3–7, Supplementary Fig. 3). According to results of direct cross-site evaluation, the base GCN model demonstrated the strongest stability and robustness for different SCZ prevalence. The mean cross-site AUCs for the five HC/SCZ ratios were GCN 0.780 (±0.015) vs. LR 0.757 (±0.029), and mean ACCs were GCN 0.772 (±0.037) vs. LR 0.723 (±0.042).

Performance stability of different EEG graph representations: In addition to the simple left-right symmetric graph structure used in our base model (Fig. 2a), we also tested other structures with more complicated network connections between channels as shown in Fig. 2b, c. We also explored adjusting the edge weights between node pairs (channel pairs) dynamically during model training. However, models obtained with these complicated graph structures dropped the cross-site AUC for about 3% compared to the simple graph structure. One potential reason is that complex EEG networks lead to an overfitting and reduce the generalizability of the diagnosis model.

Performance stability of TL strategies: steady performance improvements can be observed by incrementally implementing different TL strategies (Fig. 3b), with larger performance gains achieved by adding TUH (mean AUC: 0.787 vs. 0.824; mean ACC: 0.790 vs. 0.825). Overall, the proposed EEG_TL framework produced stable performance for different SCZ prevalence, with AUCs of 0.793–0.852 and accuracies of 0.786–0.858, respectively. An average improvement of 4.4% for AUC and 5.3% for ACC was achieved compared to the base model; and an average improvement of 4.6~25.21% for AUC and 7.9~24.35% was achieved for ACC compared to other implemented methods. More detailed discussion of performance contribution from each TL strategy can be found in the Discussion section.

In addition, Moscow’s performance is reported in Supplementary Table 11. Notably, the Moscow dataset belongs to a different age group (adolescent only), ethnicity (Russia vs. China vs. the USA), gender (boy only), and length of EEG samples (56 s vs. 2 min) from that of Chengdu and TUH. Applying the base diagnosis model to cross-site data directly yielded an AUC of 0.606 and an ACC of 0.595, relatively lower than the cross-site performance of Hangzhou (0.798/0.798). Encouragingly, an improvement of ~10% was obtained by EEG_TL (0.702/0.690), demonstrating the potential of using TL strategies to scale up the dissemination of EEG-based diagnosis tools for SCZ.

Feature significance and generalization in HC and SCZ discrimination

Supplementary Fig. 6 and the section of Feature distribution comparison of HCs depict example feature distributions of HCs in the three datasets (i.e., TUH, Chengdu, Hangzhou). In addition, Fig. 4 visualizes feature comparisons between the Chengdu and Hangzhou datasets from different aspects:

1. Feature correlation heatmaps. In total, 4 groups of features in 5 frequency bands and 16 EEG channels are analyzed, including EEG frequency features of power (16*5), DE (16*5), EEG time-domain features (16*14), and demographic features (i.e., gender and age). SCZs in the Chengdu dataset are highly correlated with frequency features. In contrast, SCZs in the Hangzhou dataset also have a strong correlation with time-domain features in addition to frequency features.

2. Mean EEG power in each frequency band for HCs and SCZs. Significant differences in mean power values are observed not only between HCs/SCZs in two different datasets but also between HCs/SCZs of the same dataset, indicating the essential heterogeneity in EEG signals and the necessity to select the most informative features with good differential power and generalizability.

3. shows the SHAP values of the top 15 features of the diagnosis model before and after TL, indicating the evolution of feature importance from Chengdu to Hangzhou through the TL process. Each row shows the importance of different values of a feature, with red for high feature values and blue for low feature values. The direction of the positive x-axis represents the importance of each feature value to support SCZ, and the direction of the negative axis represents the importance of each feature value to support HCs. The rows are ranked by the overall feature importance vertically.

Fig. 4. Feature comparisons between the Chengdu and Hangzhou datasets.

a Feature correlation heatmaps. In total, 4 groups of features in 5 frequency bands and 16 EEG channels are analyzed, including EEG frequency features of power (16*5), differential entropy - DE (16*5), EEG time-domain features (16*14) and demographic features (i.e., gender and age). b Mean EEG power in each frequency band for healthy controls (HCs) and schizophrenia patients (SCZs). Significant differences of mean power values are observed not only between HCs and SCZs of the same dataset but also between HCs/SCZs in two different datasets. c Feature importance evolution from Chengdu to Hangzhou via transfer learning. The ranking of feature importance changes significantly from Chengdu to Hangzhou, but the most important features lie in the theta (θ, 4–8 Hz) and alpha (α, 8–3 Hz) power in both datasets. The ratio between HCs and SCZs is 1:1 in both datasets.

As shown in Fig. 4c, salient features of Chengdu mainly consist of frequency features, while time-domain features (e.g., energy of time sequence) are among the top features in Hangzhou in addition to frequency features, which is aligned with the feature correlation shown in Fig. 4a, indicating the adaptation from Chengdu to Hangzhou. Despite significant changes in feature importance ranking between the Chengdu and Hangzhou datasets, it is interesting to note that the most important features remained in the theta and alpha band power in both datasets. From a spatial perspective, we observed 5 common channels (Fp1, Fp2, P4, O1, and O2) in the top 15 features in both datasets, which may be related to essential heterogeneity. Although the consistency of spatial features is not as good as that of temporal features, it still suggests the existence of some common spatial-temporal EEG abnormality underlying SCZ pathology. Upon closer examination of the salient feature sets of Chengdu and Hangzhou, we found that different brain regions of the same frequency band showed varying degree of importance in these two datasets (e.g., the alpha band power of the F4 channel vs. the O2 channel, the theta band power of the Fp1 channel vs. the O2 channel). In addition to the theta and alpha band power that demonstrated importance globally across different datasets, these findings suggest that different brain regions of the same frequency band may play different roles in distinguishing SCZ patients from healthy controls, highlighting the potential for personalized diagnosis and treatment based on individual differences in brain function.

Discussion

There is an urgent need for accurate and timely screening/diagnosis of SCZ globally, especially with objective and cost-effective assessments that can be widely disseminated in real-world settings [2, 3]. This study takes the initiative to investigate the cross-site transportability of EEG-based diagnosis models and the feasibility of employing TL techniques to improve the transportability. Overall, an AUC of 0.780 (±0.015) and an ACC of 0.772 (±0.037) were obtained for different SCZ prevalence in direct cross-site evaluation. The performance was further improved to an AUC of 0.824 (±0.027) and an ACC 0.825 (±0.035) with TL strategies (Supplementary Fig. 2 and Supplementary Tables 3–7). Stable performance was produced in cross-site evaluation for different SCZ prevalence, indicating great potential in real clinical scenarios. Despite feature importance varied significantly among different datasets, EEG theta and alpha power appeared to be the most significant and translational biomarkers of SCZ pathology.

Interestingly, a combination of a simplified EEG graph structure and a set of semantically interpretable EEG features in DL-based models achieved the optimal transportability for SCZ diagnosis in this study. These results are encouraging for real-world applications and are aligned with findings on other disorders for cross-site generalizability [12, 50, 51]. It is important to note that EEG activity changes dynamically as the human brain develops from adolescence to adulthood [52]. Consequently, it is not surprising that when our model trained on adult data was applied to the adolescent Moscow dataset, its performance decreased. Additionally, medication may be a potential confounding factor. Patients in the Moscow adolescent dataset may be first-episode or in early stages, whereas the Chinese dataset consists of chronic inpatients, the majority of whom were receiving medication (Supplementary Table 12). These differences in both developmental stages and medication status may have contributed to the observed discrepancy in performance when the model was applied to the adolescent dataset. Encouragingly, an improvement of ~10% was obtained with TL strategies, demonstrating the potential to scale up the dissemination of EEG-based diagnosis tools for SCZ. Future research with enriched adolescent data will be conducted to further address this discrepancy.

Although previous studies have attempted to assist SCZ diagnosis through PET, MRI and fMRI, they have limitations in sample size, cost, promotion and feasibility in clinical practice. On the other hand, as a potentially cost-effective solution with wide dissemination, EEG is still rarely examined in large-scale research, which may be due not only to lack of information, but also to methodological problems. EEG is often highly contaminated with artifacts, and the quality of EEG data of participants varies greatly. Here, we applied a standardized preprocessing method, namely Automagic method [40], which has been proved to be effective in reducing a large number of EEG artifacts automatically. Based on relatively large clinical EEG datasets from multiple sites, we have the opportunity to develop more advanced DL methods and provide new insights into SCZ pathology.

Despite the significant challenges in identifying effective biomarkers, they hold the potential to not only aid in the diagnosis, prognosis, risk stratification, and treatment monitoring of schizophrenia (SCZ) in clinical research, but also to provide insights into the biological foundations of the disorder. While psychiatric interview and examination can distinguish schizophrenia patients in clinical practice in some degree, EEG offers added value in understanding the underlying pathology of SCZ and guiding the monitoring of treatment response. Of note, the predictive value of a biomarker depends on its generalizability across clinical population, as well as its sensitivity and specificity for the target population. In our study, although the top 15 features of Chengdu and Hangzhou datasets are different, theta and alpha power disruptions appeared to be common in both datasets, suggesting that they are potential biomarkers of SCZ pathology. It is worth noting that previous studies have demonstrated that the abnormal EEG power of SCZ is not related to disease treatment or disease duration [53]. On the other hand, delta/theta power in patients with SCZ showed a significant positive correlation with frontal metabolism, while alpha power exhibited a significant positive correlation with subcortical metabolism. The positive association between low-frequency power in SCZ and frontal-subcortical metabolism [54] might be supported by the observed relationship between low-frequency power enhancement, negative symptoms, and ocular motor dysfunction. Furthermore, the disruption of thalamic-cortical networks could contribute to the symptomatology and information processing defects observed in patients with SCZ [55]. In summary, the clinical and biological correlates of low-frequency power abnormalities (theta and alpha) likely indicate the presence of thalamic and frontal-lobe dysfunction in SCZ. Furthermore, low-frequency oscillations are associated with long-distance brain communication[62], which is particularly important when conceptualizing schizophrenia as a dysconnectivity syndrome[63,64]. The dysconnection hypothesis in schizophrenia proposes a failure of functional integration within distributed but circumscribed neuronal systems, which may manifest as reduced functional connectivity compared to healthy controls[65]. Consequently, disruption of theta and alpha activities may thus establish a link between the symptoms and signs of schizophrenia and the dysconnection hypothesis. Additionally, further research should be undertaken on antipsychotic-naïve patients to examine whether EEG theta and alpha power can be utilized as potential biomarkers for predicting treatment response.

Translational ability and scalability of the proposed framework: (1) Given the limited sample sizes, population heterogeneity, and sampling variations, there is always a need of TL techniques for cross-site transportability. By integrating multiple effective TL strategies, the framework of EEG_TL is composable and flexible to adapt to various clinical settings with different resources. With the help of large-scale and low-cost external resources, the model generalizability could be improved by enriched EEG representations. Diagnosis models can be applied directly to cross-site data, or first adjusted using inexpensive, unlabeled data from new sites if required. (2) Previous studies using MRI data have shown that hybrid data from multi sources can increase model generalization on new datasets [56]. Our proposed framework is highly scalable with additional data sources. Domain discriminator can also be used in the pre-training stage to generalize feature representations from multi-source inputs for better performance.

Limitations and future work: (1) The experiments in this study used data from four sites. Although transportability across different population distributions were examined, the model needs to be validated in a next step on more samples from different settings. (2) Currently, EEG signals of sixteen channels commonly present in four resources were employed in this study. For the convivence of EEG signal sampling using wearable devices, performance and generalizability using reduced number (e.g., one or two) of channels will be explored in the next step. (3) Moreover, the pre-configuration of external source ratios for data augmentation can be flexible in different clinical scenarios. (4) The proposed method can be easily generalized to other clinical diseases (such as depression and Alzheimer’s disease) and EEG-based applications (such as prognosis prediction). This has important implications for facilitating automated diagnosis of neurological and psychiatric disorders by leveraging large-scale, low-cost data resources. (5) Explainable Artificial Intelligence (XAI) comprises a collection of processes and techniques that enable human users to understand and trust the results and output generated by machine learning algorithms. While we have attempted to interpret our model using Sharp values in the present study, we acknowledge the value of incorporating XAI in future research. The approach will help illuminate the relevance of specific characteristics in the final model, further enhancing its trustworthiness and transparency.

In summary, this study investigated whether large-scale public EEG data from external domains and small-scale EEG data from different mental health sites can be leveraged to produce interpretable and translational models. Promising cross-site performance demonstrated the feasibility of using EEG biomarkers for differential diagnosis. The proposed framework is compositional and flexible to adapt to different clinical practice scenarios, and it can also be generalized to other clinical applications using EEG biomarkers.

Author contributions

The work presented here was carried out in collaboration among all authors. HJ, YZ, and TL designed the initial methods and experiments. HJ, LZ, QW, XL, WD, ZG, FH, SH, and TL discussed and refined the study design. CL and RX carried out the data collection, HJ, PC and YZ analyzed the data, HJ and YZ interpreted the results and drafted the paper. All authors have attributed to, read, and approved the manuscript.

Funding

This work was partly supported by STI2030-Major Projects (grant number 2022ZD0212400, 2021ZD0200404), the Fundamental Research Funds for the Central Universities (grant number 226-2022-00138), Hangzhou Biomedical and Health Industry Special Projects for Science and Technology (2021WJCY240), the National Natural Science Foundation of China (grant number 81920108018), the Key R & D Program of Zhejiang (grant number 2022C03096), Project for Hangzhou Medical Disciplines of Excellence.

Data availability

The TUH dataset can be open-accessed at https://isip.piconepress.com/projects/tuh_eeg/html/downloads.shtml, and the Moscow dataset can be open-accessed at http://brain.bio.msu.ru/eeg_schizophrenia.htm. The Chengdu/Hangzhou clinical EEG datasets supporting the findings of this study are accessible from the corresponding author upon reasonable request, with the approval of the Institutional Review Board of the West China Hospital/the Hangzhou Seventh People’s Hospital.

Code availability

The codes of the proposed approach are accessible at https://github.com/ChenPeiyin/Diagnosis-based-EEG-A-cross-site-study. Codes of traditional machine learning algorithms used in this study are publicly available at https://github.com/scikit-learn/scikit-learn. The code used for SHAP analysis is available at https://github.com/slundberg/shap.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41386-023-01658-5.