Identification of major depression patients using machine learning models based on heart rate variability during sleep stages for pre-hospital screening

Abstract

With the COVID-19 pandemic causing challenges in hospital admissions globally, the role of home health monitoring in aiding the diagnosis of mental health disorders has become increasingly important. This paper proposes an interpretable machine learning solution to optimise initial screening for major depressive disorder (MDD) in both male and female patients. The data is from the Stanford Technical Analysis and Sleep Genome Study (STAGES). We analyzed 5-min short-term electrocardiogram (ECG) signals during nighttime sleep stages of 40 MDD patients and 40 healthy controls, with a 1:1 gender ratio. After preprocessing, we calculated the time-frequency parameters of heart rate variability (HRV) based on the ECG signals and used common machine learning algorithms for classification, along with feature importance analysis for global decision analysis. Ultimately, the Bayesian optimised extremely randomized trees classifier (BO-ERTC) showed the best performance on this dataset (accuracy 86.32%, specificity 86.49%, sensitivity 85.85%, F1-score 0.86). By using feature importance analysis on the cases confirmed by BO-ERTC, we found that gender is one of the most important factors affecting the prediction of the model, which should not be overlooked in our assisted diagnosis. This method can be embedded in portable ECG monitoring systems and is consistent with the literature results.

1. Introduction

In the past decade, research has revealed the potential of ECG signals as a biometric feature [[1], [2], [3], [4]]. The diverse, inherent life-detection capabilities, sustained availability, and intrinsic stealth of ECG make it an intriguing biometric modality. These characteristics support the development of new applications with non-invasiveness being the key factor. Hence, there is a widespread interest in integrating ML algorithms with ECG signals to expand their application scope. Disease classification and biomarker discovery have become increasingly important in modern biological and medical research [5]. Depression remains the second leading cause of human death, with high rates of misdiagnosis and underdiagnosis [6]. Over 46% of people with mental illness are diagnosed with depression at the time of suicide, making it the most common condition. Patients with depression often suffer from other physical illnesses, which poses challenges in diagnosis and a lack of direct and effective medical guidance. Given its high global prevalence and known consequences of untreated depression, primary care clinicians have a crucial role to play in effectively diagnosing and treating depression [7]. Thus, an objective, accessible, and safe way is needed to assist in the pre-hospital screening of depression.

Since the outbreak of COVID-19, hundreds of millions of people worldwide have suffered from insomnia, with 70 million new people suffering from depression [6]. The bidirectional relationship between depression and insomnia is well-established, with insomnia being a common symptom and risk factor for depression [8]. Insomnia patients are more prone to developing depression, and depression patients experience less rapid eye movement sleep stage, which is also the target for antidepressant medication to improve sleep architecture. These all indicate a profound connection between sleep and depression at multiple levels [9]. At the same time, previous studies have shown that ECG signals can be used to reflect the degree of insomnia and assess sleep architecture with good results [[10], [11], [12]]. On the other hand, depression patients have dysfunctional autonomic nervous system (ANS) and lower HRV parameters, which are currently considered as biological phenotypes and potential regulatory factors for distinguishing ANS imbalance, depression, and other stress-related states and health conditions [13]. Compared to healthy individuals, depression patients have lower parasympathetic regulation and HRV parameters, including the root mean square of successive differences between adjacent NN intervals and high frequency (HF) power [14]. In addition to these factors, this study also considers that MDD patients have relatively stable emotions during sleep and are less susceptible to signal interference during the data collection process. Indeed, previous studies have shown that there are differences in HRV parameters between MDD patients and healthy controls. Kwon et al. found significant differences in HRV parameters between MDD patients and controls during the rapid eye movement period, with MDD patients having significantly higherα1 values and lower heart rate complexity [14]. Therefore, based on these findings, this paper proposes a method to identify MDD patients using ECG signals during a 5-min sleep period.

Exploration of existing studies revealed significant differences in HRV parameters between males and females, with female oestrogen secretion leading to differences in parameters such as low frequency (LF), HF and LF to HF power ratio (LF/HF) in females and males: at the same age, males exhibit higher LF and lower HF than females [15]. There were also gender differences in HRV levels in depressed patients, with depression being associated with higher levels of HRV in the female group and lower levels of HRV in the male group [16,17]. Using ECG signals collected in a standard cardiovascular reflex test (Ewing test) and based on Bayesian networks, Kuang et al. developed a method to differentiate depressed patients from healthy individuals using HRV sequences with an accuracy of 86.4%, including only women [18]. Byun et al. used a SVM classifier as part of a machine learning approach to classify patients with MDD from healthy individuals using HRV indicator parameters with an accuracy of 74.4% [19]. It can be seen that previous studies identifying MDD have not taken into account the effect of gender differences, lacked work on identifying male patients with MDD based on ECG signals, and had relatively low accuracy with adequate sample sizes. In this study, the difference in HRV due to gender was included as an influencing factor, and after adding gender features and processing, the short-term HRV parameters calculated from the ECG signal during the sleep phase was able to identify patients with MDD across gender better. Our method validates the possibility of identifying patients with MDD by calculating the short-term HRV during the sleep phase.

The main contributions and findings are listed below.

• •A method for adding gender characteristics as input to the categorical identification of MDD, improving the accuracy of identifying people with MDD in the population.
• •Aids in the diagnosis of depression using any 5-min short HRV parameter during the sleep phase, making it more suitable for home screening.
• •Algorithm uses ERTC based on Bayesian optimization
• •Explanatory model based on decision trees, low computational effort, validated and easy to integrate into ECG signal wearable devices

The remainder of the paper is organised as follows. Section 2 briefly describes the dataset used in the paper. The BO-ERTC depression detection system proposed in this study is presented in Section 3. Next, Section 4 describes the experimental framework, followed by the experimental results in Section 5. Finally, conclusions and future work are summarized in Section 6.

2. Material and methods

2.1. Data and materials

In this study, all polysomnographic (PSG) signal data were obtained from the STAGES and depressed subjects were diagnosed based on the PHQ-9 test, known as the Patient Health Questionnaire Depression Scale, which is a recommended and medically validated depression screening test by the American Psychiatric Association and has been widely used in major hospitals all over the world [20,21]. To balance the data, 80 subjects with a complete PSG signal were randomly selected (40 patients with MDD and 40 healthy subjects as controls, with 1:1 male to female ratio). All subjects underwent a PHQ-9 questionnaire to determine the severity of their depression, with PHQ-10 scores between 0 and 4 being considered as controls without depression and those in the 20–27 range generally being considered as having major depression. It is worth mentioning that the STAGES data do not involve the medication use of patients with MDD. All data were approved by the local institutional review board, and each participant provided written informed consent before participation. The HRV fragments used and the information composition of the subjects are shown in Table 1 .

Table 1.

Corresponding statistical information of subjects.

Sexual	Class	Subjects	HRV Segments	Average Age	Average Score of PHQ-9
Male	MDD	20	1965	32.21	24.32
	Health	20	2000	28.13	2.25
Female	MDD	20	2517	33.81	22.16
	Health	20	2318	29.46	2.01

We screened the PSG signals in STAGES, using only data that matched at least 2 h of total sleep time. We then screened the PSG signal segments from the time the subject fell asleep to full wakefulness based on the sleep period labels in the STAGES dataset. The raw ECG signal data of the ECG channels were then extracted from the multi-channel PSG signals throughout the sleep session of the night, and the signals were down-sampled to 200Hz to reduce the amount of computation while ensuring the integrity of the ECG signal as much as possible. It is known that the impedance between the electrodes measuring the ECG signal and the skin is relatively displaced with the change of the motion state, and the impedance between the two changes with the relative displacement, resulting in the detected ECG signal in motion artefacts appear, whereas in this experiment motion artefacts and ectopic beats in the sleep phase were less compared to the other experimental states, as shown in Fig. 1 . Therefore, in order to further reduce the computational effort, the artefacts were not removed for the time being, considering that the method is expected to be integrated into a wearable device at a later stage. The overall pre-processing process is as follows: first, the raw ECG signal is passed through a fifth-order 0.5 Hz high-pass Butterworth filter, followed by a trap to remove the 50 Hz I.F. interference. Afterwards, the baseline drift is removed using a wavelet resolution decomposition method, which is more suitable for the pre-processing of ECG signals. Specifically, the wavelet transform has the ability to characterise local features of the signal and the resolution is adaptive in the low and HF bands, with lower temporal resolution and higher frequency resolution in the LF part and higher temporal resolution and lower frequency resolution in the HF part, making it suitable for analysing non-stationary signals, extracting local features of the signal and removing baseline drift. The baseline drift of the ECG signal is lower than 1 Hz, but it also contains useful signals at lower frequencies as well, with the ST segment in the frequency range of 0.7–2.0 Hz and the P and T waves in the frequency range of 0.7–10 Hz [22]. Therefore, how to remove the baseline drift while ensuring that the useful signal is not compromised is the key consideration when denoising. This noise in the lowest frequency band is in the higher scale sub-bands during wavelet decomposition. With the provided signal down-sampled to 200 Hz, the frequency ranges of the sub-band coefficients at each scale are shown in Table 2 when assuming an 8-layer wavelet decomposition

Fig. 1.

Schematic diagram of proposed algorithm.

Table 2.

The frequency range of sub-band coefficients at each scale.

Sub-band coefficients	Frequency ranges (Hz)
cd1	50–100
cd2	25–50
cd3	12.5–25
cd4	6.25–12.5
cd5	3.125–6.25
cd6	1.5625–3.125
cd7	0.78125–1.5625
cd8	0.390625–0.78125
ca8	0–0.390625

Combined with Table 2, and based on the baseline drift noise and the frequency range of the LF band in the ECG signal, the frequency range of the baseline drift can be determined as 0–0.39Hz, which is the frequency range covered by the ca8 sub-band, so the decomposition scale can be determined as j = 8.

The recommended length for HRV analysis is 24 h for long-term and 5 min for short-term monitoring [23]. Using 24-h HRV analysis, the physiological patterns reflected in the overall HR changes, including diurnal differences, can be monitored as subjects carry out their normal daily activities during the HRV recording period. Long-term methods have been used to assess cardiovascular mortality and patient prognosis or early diagnosis [23]. However, this analysis is difficult and less reproducible. In contrast, short-term recordings, 5-min HRV, are considered methodologically adequate. Therefore, we segmented the processed ECG signals of the entire overnight sleep stage into non-overlapping 5-min segments. It is worth noting that the term "sleep stage" used in this study does not refer to the traditional definition of sleep stages. In contrast to the traditional definition, the term "sleep stage" in this study refers to the entire duration from falling asleep to fully awakening, during which any 5-min signal recorded by the ECG signal acquisition device after falling asleep can be used to greatly simplify the difficulty of ECG signal recording. With only a small amount of data, we can obtain sufficient information for subsequent analysis without excessive time and resource costs. Therefore, it is necessary to clarify the difference between this definition and the traditional definition.

Fig. 1 illustrates a proposed system for classifying HRV parameters of sleep stages in MDD patients and control group. As shown in the figure, the system consists of four steps. In the proposed system, the first step is to pre-process the ECG signal and divide it into non-overlapping 5-min segments. Then, the time-domain and frequency-domain parameters of HRV are calculated for each segment and annotated. Next, these feature vectors are input into BO-ERTC and BO-SVM classifiers. Finally, the proposed model is trained and tested using 10-fold cross-validation to compare and analyse the performance of the selected classifiers.

3. Data processing

3.1. HRV parameters extraction

HRV, which analyses the subtle temporal variations and patterns from one cardiac cycle to the next, has become the accepted terminology for defining instantaneous heart rate and inter rhythm variability. The complex and variable heart rate is an indicator of the regulatory system and these subtle variations are often difficult to measure or negligible on a conventional ECG recorded on the body surface. HRV parameters are responsible for most of the indicators of heart rate abnormalities. The common methods of analysis for HRV are generally time-domain, frequency-domain and nonlinear analysis, among others.

New methods based on chaos theory, such as fractal dimension, approximate entropy, detrended fluctuation analysis, Lyapunov exponents, and symbolic analysis, have been developed and applied in various fields. Non-linear dynamical methods for HRV analysis can provide a more sensitive way to characterise the function or dysfunction of autonomous control mechanisms. To date, chaos analysis for the evaluation of autonomic regulation has been studied in normal subjects and patients with cardiovascular disease [24,25]. However, these techniques are mathematically complex and require more powerful calculations, whereas this method focuses more on the simplicity and generality of the acquisition and HRV parameter calculations. In addition, they are still under development and evaluation, so this method has been chosen to extract only HRV time and frequency domain parameters based on the "NeuroKit2" toolbox [26].

3.1.1. Calculation of time domain parameters

The results of the time domain analysis can be used as a general assessment of the role of the ANS in heart rate regulation. Time domain analysis includes the difference between the mean heart rate interval, the mean heart rate, the maximum and minimum heart rate and parameters such as HRV_SDNN, HRV_RMSSD based on RR interval calculation, the standard deviation of the continuous difference between RR intervals (HRV_SDSD), the percentage of intervals greater than 50 Ms (HRV_pNN50) and the mean of RR intervals. Of these parameters, SDNN reflects the activities of sympathetic and parasympathetic, while HRV_RMSSD, HRV_SDSD and HRV_pNN50 are sensitive to parasympathetic modulation. However, time domain measurements have limitations in adequately quantifying autonomic dynamics and determining the rhythmic or oscillatory activity generated by different physiological control systems, hence the need to include the calculation of frequency domain parameters.

3.1.2. Calculation of frequency domain parameters

Frequency domain analysis takes a time series signal of the RR interval, captures the time and frequency through a time-frequency analysis method, and quantifies the instantaneous power spectral density at any given time by evaluating the spectrum over a short moving window [9]. The frequency domain features represent the various spectral components, namely very LF, LF, HF, and very HF. Frequency domain parameters in HF and LF are parasympathetic and sympathetic, respectively. The ratio of HRV_LF to HRV_HF power (HRV_LFHF) is also often used as an indicator of sympathetic vagal balance, a measure of the relative contribution of the SNS to PNS activity. The HRV_HF component is relatively well established as an indicator of HRV parasympathetic modulation, and under controlled conditions, HRV_LnHF (The log transformed HF) is often used as an estimate of vagal tone [27,28]. The parameters extracted for all calculations are shown in Table 3 .

Table 3.

All parameter.23 HRV features were extracted from each phase.

Domain	Parameter	Description
Time	HRV_MeanNN	The mean of the RR intervals.
	HRV_SDNN	The standard deviation of the RR intervals.
	HRV_SDANN1	The standard deviation of average RR intervals extracted from 1-min segments of time series data.
	HRV_RMSSD	The square root of the mean of the squared successive differences between adjacent RR intervals. It is equivalent (although on another scale) to SD1, and therefore it is redundant to report correlations with both.
	HRV_SDSD	The standard deviation of the successive differences between RR intervals.
	HRV_CVNN	The standard deviation of the RR intervals (SDNN) divided by the mean of the RR intervals (MeanNN).
	HRV_CVSD	The root mean square of successive differences divided by the mean of the RR intervals (MeanNN).
	HRV_MedianNN	The median of the RR intervals.
	HRV_MadNN	The median absolute deviation of the RR intervals.
	HRV_MCVNN	The median absolute deviation of the RR intervals (MadNN) divided by the median of the RR intervals (MedianNN).
	HRV_IQRNN	The interquartile range (IQR) of the RR intervals.
	HRV_pNN50	The proportion of RR intervals greater than 50 ms, out of the total number of RR intervals.
	HRV_pNN20	The proportion of RR intervals greater than 20 ms, out of the total number of RR intervals.
	HRV_HTI	The HRV triangular index, measuring the total number of RR intervals divided by the height of the RR intervals histogram.
	HRV_TINN	A geometrical parameter of the HRV, or more specifically, the baseline width of the RR intervals distribution obtained by triangular interpolation, where the error of least squares determines the triangle. It is an approximation of the RR interval distribution.
Frequency	HRV_VLF	The spectral power of very low frequencies (by default, .0033 to .04 Hz).
	HRV_LF	The spectral power of low frequencies (by default, .04 to .15 Hz).
	HRV_HF	The spectral power of high frequencies (by default, .15 to .4 Hz).
	HRV_VHF	The spectral power of very high frequencies (by default, .4 to .5 Hz).
	HRV_LFHF	The ratio obtained by dividing the LF power by the HF power.
	HRV_LFn	The normalized LF, obtained by dividing the LF power by the total power.
	HRV_HFn	The normalized HF, obtained by dividing the HF power by the total power.
	HRV_LnHF	The log transformed HF.

3.2. Treatment of gender differences

In view of the gender difference in HRV parameter mentioned above, most machine learning models understand integers as not text, so converting text strings like the categorical variable "gender" into numeric variables is a necessary step so that the machine learning model can calculate the correlation between features and outcomes and make correct predictions. Using one-hot encoding, which is a common and efficient encoding scheme for classification tasks, each gender classification value will be converted to a new column and the label value will be converted to a numeric form (1 or 0).

After using one-hot encoding for gender, gender itself is an input feature, and after one-hot encoding, it becomes a two-dimensional feature of men and women. This will make each category more comparable and avoid prioritization [29], The value of discrete features is extended to Euclidean space, and then the gender difference is considered as the input.

4. Methodology

4.1. Model training groups

From the perspective of machine learning, the recognition and analysis of MDD can be viewed as a regression or classification problem. In this paper, we extracted 24 features to establish machine learning models. Using these extracted features, we established a machine learning model following the process of determining the classification target (depression and healthy) based on the input features. Before training, different machine learning algorithms, such as ordinary SVM, ERTC, Bayesian network, and extreme gradient boosting, were used to model on a smaller dataset. The results showed that ERTC and SVM provided better results than all the algorithms we tried.

Therefore, ERTC and SVM were used to perform the final screening on the final dataset. We trained the ERTC and SVM with Bayesian optimization on the control group with and without gender features, respectively.

4.2. Introduction to the model

4.2.1.

ERTC is an integrated learning technique, for classifying data based on ensemble learning of decision trees (DT), which aggregates the results of multiple de-correlated decision trees collected in a forest to output classification results. Each decision tree in a forest of extremely random trees is constructed from the original training samples. At each test node, each tree has a random sample of k features, and each decision tree must select the best features from these feature sets and then split the data according to the Gini index. This random sample of features leads to the creation of multiple uncorrelated DT [30].

4.2.2. SVM

The SVM classifier is a supervised learning approach for separating two classes by finding the best separation hyperplane in the feature space [31]. For N training samples $\left\{(x_i,y_i),i=1,\cdots ,N\right\}$, where $x_i$ is the $i$ the input vector and $y_i$ is the known target, SVM training is the same as figuring out how to solve the following optimization problem:

$$\underset{\mathit{w},\mathit{b},\mathit{\xi }}{\mathit{min}}\mathit{J}\left(\mathit{w},\mathit{\xi }\right)=\frac{\mathbf{1}}{\mathbf{2}}{\mathit{w}}^{\mathit{T}}\mathit{w}+\mathit{c}\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\mathit{\xi }}_{\mathit{i}}$$ (1)

subject to:

$${\mathit{y}}_{\mathit{i}}\left[{\mathit{w}}^{\mathit{T}}\mathit{\phi }\left({\mathit{x}}_{\mathit{i}}\right)+\mathit{b}\right]\ge \mathbf{1}-{\mathit{\xi }}_{\mathit{i}},{\mathit{\xi }}_{\mathit{i}}\ge \mathbf{0}$$ (2)

Where ${\xi }_i$ is slack variable, indicating the tolerance of misclassification C is a punishment parameter that is used to penalize mistakes during training, $b$ is a bias term, $w$ is the weight applied for input data $x_i$. The kernel function $\phi \left(x\right)$ is a nonlinear transformation function that maps the input vectors into a high-dimensional feature space [32].

4.3. Model optimization and tuning

This study uses Bayes' theorem to optimise ERTC. The basic idea of parameter optimization is to use Bayes' theorem to estimate the posterior distribution of the objective function based on the data results, and then to select the combination of hyperparameters for the next sampling based on the distribution. Full consideration of the information from the previous sampling point allows the shape of the objective function to be fully learned and the parameters that orient the results towards the global maximum lift to be found. Consider the following parameters for optimising an extreme random tree: n estimators, max depth and min samples split, and the following parameter choices for a support vector machine: C, kernel, gamma; we know that using different hyperparameters for classification will give different results, and Bayesian optimization assumes a functional relationship between the hyperparameters and the loss function that the final model is to optimise There exists a functional relationship between the hyperparameters and the final loss function to be optimised by the model. The general idea of optimization for each of these parameters is as follows.

Suppose we have a function $f:x\to R$, we need find a function:

$$x^{*}=\begin{array}{c}argminf\left(x\right)\\ x\in X\end{array}$$

Bayesian optimization assumes that the model space searched is a Gaussian distribution and uses a Gaussian process to compute each time, in an iterative fashion, a new optimal parameter that is expected to improve over the currently existing parameters. We optimise the parameters of ERTC and SVM based on ten-fold cross validation of the training data.

Table 4 shows the optimal values of the hyperparameters in the ERTC and the SVM classifier on the training set after Bayesian optimization training for this dataset.

Table 4.

Hyperparameters of the two classifiers.

BO-ERTC	Group	Target	N_estimators	Max_depth	Min_samples_split
	Control group	0.942	183.6	2.231	4.698
	With gender features group	0.9395	246.2	5.07	2.551
BO-SVM	Group	Target	C	Kernel	Gamma
	Control group	0.791	100	rbf	0.0001
	With gender features group	0.798	100	rbf	0.0001

4.4. Model results and evaluation

The performance of the proposed approach is evaluated through a qualitative assessment of eight commonly used statistical measures: accuracy, sensitivity, specificity, precision, F1-score, false positive rate, and receiver operator characteristic (ROC) curve.

The accuracy indicates the ability of classifier to accurately distinguish between classes. It is expressed mathematically as per Eq. (3).

$$Accuracy=\left(\frac{TP+TN}{TP+TN+FP+FN}\right)$$ (3)

Sensitivity also called true positive rate (TPR) or recall, indicates the ability of the classifier to correctly identify true positive classes. It is computed using Eq. (4).

$$Recall=\left(\frac{TP}{TP+FN}\right)$$ (4)

Precision or positive predictive value (PPV) reflects the proportion of positive samples correctly identified to the total number of positive classes and derived using Eq. (5).

$$PPV=\left(\frac{TP}{TP+FP}\right)$$ (5)

Specificity also known as a true negative rate (TNR), represents the ability of the classifier to correctly distinguish true negative classes. It is computed using Eq. (6).

$$TNR=\left(\frac{TN}{TN+FP}\right)$$ (6)

F1-score reflects the number of samples that are correctly identified by the classifier. It is calculated using Eq. (7).

$$F1=(1+{\beta }^2)\frac{Precision\bullet Recall}{{\beta }^2\bullet Precision+Recall}$$ (7)

where:

True positive (TP) denotes the number of correctly categorized positive samples. True negative (TN) represents the number of correctly categorized negative samples. False positive (FP) represents the number of negative samples that were misclassified by the model as positive samples. False negative (FN) shows the number of positive samples.

Final, we applied a ROC curve analysis, which is built by the value of the true positive rate and false positive rate, to this optimal model. The area under the ROC curve (AUC) was also evaluated for each case.

To evaluate the model quality of BO-ERTC and BO-SVM, 10-fold cross-validation [33,34] was ultimately employed to avoid chance findings. Specifically, 80% of all data, including HRV parameters and feature information of the sleep stages of 32 subjects, were used as the training set, while the remaining 20% (see Fig. 2 ), comprising HRV parameters and feature information of the sleep stages of 8 subjects, were used as the testing set. This method divided all segments of the training set into ten parts: nine for building the network and one for validating the quality of the network. This was repeated ten times until all segments were used to validate the classifier. The final training recognition result was obtained as the average of these ten results, and Bayesian optimization hyperparameters were used based on the training results. The hyperparameters are optimised using Bayes based on the training results and finally the effect of the completed model is verified on a test set.

Fig. 2.

Schematic distribution of data.

4.5. Importance of features

Feature importance and its visualisation is an important and widely used analysis method in the field of machine learning. Due to the simplicity and interpretability of feature ranking or risk analysis, it is particularly used in fields such as biomedicine and social sciences [35]. The importance and ranking of features are found based on the coefficient value of each feature [36,37].

4.5.1. Importance analysis based on mean impurity reduction (MDI) characteristics

In constructing the BO-ERTC, for each feature, a normalized total reduction of the Gini index used to segment the feature decisions is calculated, a value known as the importance of the Gini elements. After the Gini importance has been ranked in descending order, the top k-dimensional features can be selected as required. The quantification of impurity is derived from the Gini index of the decision tree, the depth of the feature at the decision node in the tree can be used to assess the relative importance of the feature relative to the predictability of the target variable, and the features used at the top of the tree contribute to the final prediction decision for a larger proportion of the input sample. The expected fraction of the sample they contribute can therefore be used as an estimate of the relative importance of the feature, and the proportion of the sample contributed by the feature is: combined with the reduction in impurities from splitting them to create a normalized estimate of the predictive power of the feature, and the variance of such estimates can be reduced and used in feature selection by averaging the predictive power estimates across multiple random trees.

4.5.2. Analysis of the importance of alignment characteristic

Permutation-based feature importance is also a model checking technique, which can be used for any fitted estimator in tabular data. The permutation-based feature importance can only be calculated after model fitting. Permutation based feature importance is defined as the reduction degree of model score when the value of a single feature is randomly disturbed, which destroys the relationship between feature and target. Therefore, the degree of decline in model scores indicates the degree of the model's dependence on features. We need to calculate the following parameters:

Reference score s for model m on data D

For each feature j in D:

For each k in 1 … K:

Randomly disrupt the value of column j of data set D,

Calculate the score of the model's m on the disrupted data $s_{k,j}$ ,

Calculating the importance $i_j$ of characteristics $f_j$:

$$i_j=s-\frac{1}{K}\sum _{k=1}^K{s}_{k,j}$$

5. Results

In what follows, we analyse the results of the report in two ways.

• •Performance measures for BO-SVM and BO-ERTC extracted from confusion matrices using 10-fold cross-validation.
• •For the BO-ERTC, the importance of features before and after the introduction of the gender difference treatment.

Table 5 shows the use of the BO-ERTC and BO-SVM of the evaluation results, all using a 10-fold cross-validation strategy. The results show that the model with ERTC showed greater improvement in predictive performance for this type of data, with the model trained with ERTC having an overall accuracy of 86.32%, a specificity of 79.61%, a sensitivity of 91.45% and an F1 score of 0.88. The model trained with SVM had an overall accuracy of 79.29%, a specificity of 80.24%, a sensitivity of 77.90% and F1 score of 0.75.

Table 5.

Evaluation results of the model on the test set.

Group	Classifier	TPR (%)	FPR (%)	TNR (%)	FNR (%)	Accuracy (%)	Precision (%)	F1-score
Control	BO-ERTC	89.28	10.71	75.35	24.64	83.25	82.62	0.86
	BO-SVM	77.62	22.38	80.06	19.94	79.07	72.52	0.75
With gender features group	BO-ERTC	91.45	8.56	79.61	20.39	86.32	85.48	0.88
	BO-SVM	77.90	22.10	80.24	19.76	79.29	72.77	0.75

Fig. 3, Fig. 4 show the confusion matrices of the BO-ERTC and the BO-SVM classifiers on the test set before and after processing, respectively. The confusion matrix clearly summarises the probability of correct and incorrect prediction matches for each class in the test set. Each square on the x-axis (the prediction class label) indicates the number of predictions that matches to the corresponding class on the y-axis (the actual class). Diagonal elements with grey indicate correct classifications for MDD and healthy subjects, while any non-diagonal elements are incorrect matches. Fig. 5 shows the ROC curves when using the inclusion of gender features by BO-SVM and BO-ERTC inputs, with AUC values of 0.853 and 0.948 for BO-SVM and BO-ERTC, respectively.

Fig. 3.

After adding the gender difference treatment.

Fig. 4.

Before adding the gender difference treatment.

Fig. 5.

ROC curves of the BO-SVM and BO-ERTC classifiers using the feature of gender.

Fig. 6, Fig. 7 summarize the importance distributions of the features before and after the introduction of the gender difference treatment during the training and testing of the ERTC network. Three parameters, MeanNN, MedianNN, and pNN20, showed greater differences between MDD patients and healthy individuals compared to the other parameters. After the introduction of the gender difference treatment, the importance of gender features became the fourth largest feature after MeanNN, MedianNN, and pNN20 features. During the training process, the importance of MedianNN, MeanNN, and pNN20 features also increased. There was a less significant difference in the importance of predicting the classification process. This analysis annotated highly predictive HRV signatures for the detection of MDD, showing potential usefulness for prehospital screening seeking to predict MDD patients. However, it is important to note that the amount of data provided by this dataset for MDD patients is insufficient to adequately address all questions, and further data and analysis are needed to produce a robust prediction method. Nonetheless, in the future, we hope to better understand the limitations of this approach and hope that the analysis of additional data will enable high accuracy predictions for patients with MDD or even mild depression using machine learning methods.

Fig. 6.

Feature importance analysis on the training set.

Fig. 7.

Feature importance analysis on the test set.

Subsequently, due to the high computational complexity based on the ranked feature importance, we only computed the ranked importance of features on the test set. The ranked feature importance overcomes the limitation of feature importance compared to MDI, and the computation does not bias towards high-base features. Features were more important in the classification than other features. After adding the gender difference treatment, the gender feature has the highest importance in the classification, which is consistent with the MDI-based feature importance analysis in the training set.

6. Discussion

In this study, we utilized arbitrary 5-min HRV parameters during patient sleep stages and incorporated gender as a feature. During testing, our trained BO-ERTC model successfully identified 623 MDD segments and 924 healthy segments from the HRV parameters of eight participants, totalling 1792 segments. Moreover, through feature importance analysis, we found that MeanNN, MedianNN, pNN20, and gender were the most important features, which were not previously calculated in other studies [9,18,19,[38], [39], [40]]. This finding suggests that further investigation of HRV parameters is necessary to understand their role in MDD diagnosis. These results provide insight into the potential application of machine learning in the diagnosis and treatment of MDD.

The use of HRV parameters during sleep stages can effectively identify MDD, which can be attributed to poor sleep quality in MDD patients and the fact that HRV parameters can reflect ANS dysfunction, which is closely related to sleep status. In contrast to previous studies that required experimental tasks from participants, depression, as an emotional psychological disorder, can be influenced by task participation, thereby interfering with disease identification and classification results. The data used in this study came from the sleep stages of the participants, minimizing the emotional interference as much as possible. The results were consistent with our hypothesis that HRV parameters during sleep stages can also be used for the identification of MDD patients. Compared with experimental methods, this method has better performance and less environmental restrictions. This further enriches the applicable scenarios of HRV parameters as biomarkers for MDD patients. Besides, it is worth noting that, due to the different calculation schemes used, features such as MedianNN and pNN20, which are considered important in this study, were not calculated in previous studies, that may be the reason for the high classification accuracy of this approach. This further enriches the association between autonomic nervous function and HRV parameters. Furthermore, with the introduction of gender characteristics, the classification accuracy significantly improved, which also confirms the existence of differences in HRV parameters between males and females during sleep stages. Only by considering this difference can MDD patients be screened more accurately and efficiently in the population, thus increasing the outpatient visit rate.

The previous research has shown that using HRV parameters to distinguish MDD patients can achieve good results, which are summarized in Table 6 . However, these studies did not explicitly state whether the HRV parameters used for testing and training were from the same person, which cannot be ignored in practical applications [9,18,19,[38], [39], [40]]. In addition, it should be noted that predictive performance of the study can easily be overestimated in the absence of cross-validation. Kuang et al. evaluated the predictive performance using the 10-fold cross-validation method. They achieved an accuracy of 86% using 10 features, but only female participants were involved in the experiment. In contrast, in study of Byun et al., both male and female participants were involved, but the accuracy was relatively low, only 74%. This study also evaluated the test performance using the 10-fold cross-validation method, and the results showed that the accuracy of using only HRV parameters to identify MDD patients would decrease when male participants were included, which is consistent with previous research results. In terms of identification accuracy, the average accuracy (86.32%) of this study is moderate compared to previous research. However, it should be noted that the studies of Minwoo et al. and Zhang et al. are based on high accuracy with a small number of samples, and it needs to be considered and verified whether their studies can be guaranteed to remain at the same level when the number of subjects increases. When the number of experimental subjects in this study was about 80, our method showed a significant improvement in accuracy. In terms of algorithm integrability, many machine learning algorithms now require a large amount of storage and computational resources, which have high requirements for power consumption, size, computing power, and storage capacity. In contrast, statistical learning-based algorithms have well-designed theoretical models and excellent generalization performance, and require relatively low computational complexity. However, neural network-based algorithms require processing a large number of parameters, accessing massive weight coefficients, and conducting a large amount of data exchange between on-chip RAM and external storage in a short period to complete calculations. In the studies of Minwoo et al. and Zhang et al., they used neural network-based models, and compared to this study, the model's computational and power consumption requirements were higher. The STM32 Cortex-M processor is commonly used in current electrocardiogram signal acquisition devices, and its power consumption is relatively low. This study used a model based on the decision tree theory in statistical learning, which has relatively low requirements for the processor and is easier to integrate. In terms of experimental design, this study does not require patients to undergo experiments according to a specific experimental plan, but only needs to collect HRV parameters during any 5 min of the patient's sleep, which reduces the complexity of the experiment. However, in the studies of Minwoo et al. and Zhang et al., HRV parameters obtained from a multimodal emotional content stimulation experimental task, which, despite its accuracy, requires inducing subjects to experience a variety of emotions, including pain and stress, which may cause additional psychological burden and discomfort for subjects suffering from depression. In addition, this method requires time and resources to collect data as the subjects participate in the experiment. In contrast, our method only requires ECG signals to be collected within 5 min of falling asleep, without the need for an experimentalist, greatly reducing the psychological burden on study participants and the cost of collecting experimental data. Therefore, this study achieved the transfer of depression screening from the laboratory to the home environment, which facilitated the screening of patients. However, although this study has important theoretical and practical significance, it must be acknowledged that there are certain limitations. Due to data source limitations, we could not further expand the sample size of the experiment, making it difficult to verify the robustness of this method in different populations. Therefore, to further verify the practical availability of this solution, we plan to integrate this model into embedded devices and recruit more experimental subjects to further expand the sample size and conduct verification. To address the problem of a small amount of data, we did not solely use 5-min segments of data from individual subjects as training sets. Instead, we evenly divided the data from the entire night's sleep of each subject and set these segments as labels for MDD or health, which allowed the model to better learn the differences between MDD patients and healthy individuals.

Table 6.

Summary of previous HRV-based MDD detection studies.

Authors	Subjects	Test task	Feature Selection	Classification method	ACC
Kuang et al. [18]	MDD 38 HC 38 (female)	Ewing test	Correlation-based method (10 selected)	Bayesian networks 10-fold CV	86%
Byun et al. [19]	MDD 37 HC 41	Mental arithmetic test	SVM-RFE (2 selected)	SVM LOO CV	74%
Minwoo et al. [38]	24a	Multimodal affective contents test	N/A	neuro-fuzzy network with weighted fuzzy membership functions	87%
Roh et al. [9]	23a	N/A	N/A	SVM	71%
Zhang et al. [39]	MDD 10 HC 10	Multimodal affective contents	N/A	Neuro-fuzzy network	95%
Sun et al. [40]	MDD 44 HC 47	Random number generation	N/A	Logistic regression	79%
This study	MDD 40 HC 40	Sleep phase	N/A	BO-ERTC 10-fold CV	86%

Abbreviations: HC, healthy control; N/A, not applicable; CV, cross-validation; ACC, accuracy. ^a Total number of subjects.

The aim of this study was to develop an automatic classification model based on HRV parameters derived from ECG signals collected during sleep, which can be embedded into wearable ECG monitoring devices for the identification of MDD patients, as shown in Fig. 8 . The memory requirement of our developed model was 692.867 MB, which is slightly higher than the current storage capacity of wearable ECG monitoring devices. However, our method has a low computational cost and a simple detection method. Therefore, we believe that this model can be integrated into embedded devices by expanding the storage capacity, which has certain application value and can provide a self-screening solution for MDD patients in the early stage at home and an effective reference for timely medical treatment. A surprising point is that the HRV inputs involved in our algorithm can also be obtained from ballistocardiogram (BCG) signals [41]. BCG signal collection has advantages such as better unconstrained and relatively low psychological impact on patients. This inspires us to use HRV parameters extracted from BCG signals to identify MDD patients in the future. With the rise and development of wearable devices, home health monitoring becomes possible, and we can collect and calculate physical data effectively at home, further expanding the applicability of this method. For patients with MDD, this method can provide a convenient and effective pre-hospital screening method and even assist doctors in making professional diagnoses more objectively. In conclusion, this study provides a promising method to help depression patients achieve self-screening as early as possible, and is expected to have significant clinical implications.

Fig. 8.

Proposed of the testing program.

Funding

This research was funded by the National Key R&D Program of China (2022YFC2402203) and the National Natural Science Foundation of China (52277230).

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.