A hybrid model for the classification of Autism Spectrum Disorder using Mu rhythm in EEG

Abstract

BACKGROUND:

Autism Spectrum Disorder (ASD) is a condition with social interaction, communication, and behavioral difficulties. Diagnostic methods mostly rely on subjective evaluations and can lack objectivity. In this research Machine learning (ML) and deep learning (DL) techniques are used to enhance ASD classification.

OBJECTIVE:

This study focuses on improving ASD and TD classification accuracy with a minimal number of EEG channels. ML and DL models are used with EEG data, including Mu Rhythm from the Sensory Motor Cortex (SMC) for classification.

METHODS:

Non-linear features in time and frequency domains are extracted and ML models are applied for classification. The EEG 1D data is transformed into images using Independent Component Analysis-Second Order Blind Identification (ICA-SOBI), Spectrogram, and Continuous Wavelet Transform (CWT).

RESULTS:

Stacking Classifier employed with non-linear features yields precision, recall, F1-score, and accuracy rates of 78%, 79%, 78%, and 78% respectively. Including entropy and fuzzy entropy features further improves accuracy to 81.4%. In addition, DL models, employing SOBI, CWT, and spectrogram plots, achieve precision, recall, F1-score, and accuracy of 75%, 75%, 74%, and 75% respectively. The hybrid model, which combined deep learning features from spectrogram and CWT with machine learning, exhibits prominent improvement, attained precision, recall, F1-score, and accuracy of 94%, 94%, 94%, and 94% respectively. Incorporating entropy and fuzzy entropy features further improved the accuracy to 96.9%.

CONCLUSIONS:

This study underscores the potential of ML and DL techniques in improving the classification of ASD and TD individuals, particularly when utilizing a minimal set of EEG channels.

1. Introduction

People with ASD often encounter challenges in social interaction and communication [1], particularly in grasping verbal and non-verbal cues [2]. Electroencephalography (EEG) and magnetoencephalography (MEG) are utilized to capture brain signals. They offer insights into the neural mechanisms behind ASD by identifying distinct patterns of brain activity and connectivity [3]. Early detection and appropriate interventions, such as behavioral therapies and medications, can enhance the social and communication abilities of individuals with autism, thereby enhancing their overall quality of life [4]. Difficulties in sensorimotor functions within ASD may contribute to diminished social attention in early development, leading to subsequent deficits in social, communicative, and emotional growth. These challenges not only influence non-social aspects like specialized interests but also impact social behaviors such as coordinating eye contact with speech and gestures, interpreting others’ actions, and responding appropriately [5]. The impairment of the Mirror Neuron System (MNS) could underlie sensorimotor difficulties in individuals with ASD, affecting their social communication abilities [6]. The observation of an action typically triggers a motor simulation in the observer’s MNS. Recent studies have highlighted the modulation of MNS responses during action observation by various factors [7]. One significant measure of mirror neuron activity studied in humans is mu (8–13 Hz) rhythm with suppression. During rest, synchronous firing of neurons in the sensorimotor area generates large-amplitude EEG and MEG oscillations in the mu rhythm. However, when individuals perform, imagine, or observe actions, this synchrony decreases, leading to mu-rhythm suppression, also known as mu-rhythm suppression [8]. This suppression reflects the integration of perceptual and motor representations for understanding others’ mental states, a process that might be disrupted in individuals with autism-spectrum traits [9]. The EEG measures mu rhythms, which are cortical field phenomena, associated with the somatomotor system [10] and typically manifest in the sensorimotor cortex electrodes C3, C4, and Cz. Artificial intelligence (AI) is pivotal in diagnosing and managing various neurological conditions like Alzheimer’s disease (AD), epilepsy, and attention deficit hyperactivity disorder (ADHD). AI systems analyze extensive datasets, including genetic information, medical records, and lifestyle factors, to predict Alzheimer’s risk. Additionally, AI algorithms scrutinize medical imaging data such as MRI and PET scans, detecting subtle brain changes indicative of Alzheimer’s before symptoms appear. This early detection enables personalized preventive strategies, delaying or mitigating the disease’s onset. Moreover, AI algorithms predict epileptic seizures by analyzing EEG data, facilitating the development of wearable devices that can alert patients or caregivers beforehand for timely intervention. AI-powered decision support systems assist healthcare providers in selecting optimal epilepsy treatments. Furthermore, AI-driven tools evaluate ADHD symptoms via behavioral data, aiding clinicians in monitoring severity, treatment response, and medication adherence. AI’s ability to analyze diverse datasets, including genetic, neuroimaging, and behavioral data, allows for the identification of ADHD subtypes and personalized treatment interventions. While AI holds promise for transforming neurological disorder management, rigorous validation, ethical deployment, and integration into clinical practice are essential to ensure patient care and safety. Artificial intelligence (AI) holds promise in revolutionizing ASD identification and treatment by integrating signal processing techniques with clinical expertise to develop more precise diagnostic methods. Objective approaches employing technologies like neuroimaging and computational algorithms can address the limitations of subjective assessments. While the exact relationship between mu rhythm and autism remains unclear, ongoing research endeavors to deepen our understanding of the social and cognitive impairments associated with autism.

2. Literature review

Khandaker et al. required classifying autism spectrum disorder across both children and adults, leveraging features such as basic health parameters, screening responses, scores, test types, and country of origin. They employed various classification models including Logistic Regression, Gaussian Naïve Bayes, Multinomial Naïve Bayes, Random Forest, and Support Vector Machine [11].

Liao et al. innovatively extracted features from facial expressions, eye fixation, and EEG, then a weighted naive Bayes algorithm is used for hybrid fusion. They found EEG to have the highest discriminatory power compared to behavioral data [12]. Kang et al. integrated eye-tracking and EEG data, conducting power spectrum and face gaze analysis within selected Areas Of Interest (AOI). They utilized Minimum Redundancy Maximum Relevance (MRMR) for feature selection and Support Vector Machine to distinguish between autistic and typically developing children [13]. Sinha et al. processed EEG signals using the discrete wavelet transform and extracted statistical features like mean, variance, standard deviation, kurtosis, skewness, and Shannon entropy. K-Nearest Neighbour (KNN) classifier is utilized for classification [14]. Arunkumar et al. explored spectral analysis of EEG signals using the short-time Fourier transform, for classifying autism versus normal [15].

Haputhanthri et al. processed the signal using a discrete wavelet transform and extracted statistical features from the resulting EEG. These features were then inputted into a random forest classifier [16]. Abdolzadegan et al. examined various feature extraction methods, including Wavelet Transform, Power Spectrum, Fractal Dimension, Fast Fourier Transform (FFT), Lyapunov Exponent, Correlation Dimension, Detrended Fluctuation Analysis, Entropy, and Synchronization Likelihood, in conjunction with machine learning models. They observed that the Support Vector Machine achieved higher accuracy in classification compared to KNN [17]. Subudhi et al. pre-processed EEG signals with a low pass filter and ICA, and then classified them using a Support Vector Machine (SVM) [18]. Raja and Priyab identified variations in brain EEG signals using Auto-Regressive features, achieving maximum classification accuracy with Feed Forward Neural Network (FFNN) [19]. Saran and Pirouz used machine learning algorithms including KNN, Random Forest Classifier, Support Vector Machines, and Logistic Regression to differentiate autistic and neurotypical individuals [20]. Alotaibi et al. employed functional brain connectivity and a cubic support vector machine for diagnosing ASD, achieving promising results [21].

Thabtah and Peebles proposed a machine learning framework for screening adolescents and adults with autism, achieving good performance across all metrics [22]. Mohi ud Din and Jayanthy extracted Autoregressive (AR) coefficients, Shannon entropy, Multifractal wavelet leader estimates, Multiscale wavelet variance and Discrete Fourier Transform (DFT) coefficients from EEG brain waves of ASD and normal subjects. Support Vector Machine (SVM), Decision Tree (DT), Logistic Regression (LR), k-Nearest Neighbor (k-NN) and Feed-forward Neural Network (FNN) were utilized as classification [23]. Tawhid et al., pre-processed EEG signals and undergo conversion into two-dimensional images via short-time Fourier transform. Subsequently, textural features are extracted, significant features are selected using principal component analysis, and fed into a support vector machine classifier [24]. Mohanty et al., utilized Principal Component Analysis (PCA) and Deep Neural Network (DNN) for effective ASD identification [25]. Suman et al. anticipated and analyzed ASD-related issues across different age groups using deep learning techniques and achieved promising results [26]. Radhakrishnan et al. introduced a technique for detecting ASD through EEG data, with ResNet50 outperforming other pretained architectures [27]. Mohi ud Din and Jayanthy utilized CWT and deep Convolutional Neural Networks (CNNs) for EEG signal classification, achieving better results with SqueezeNet and SVM [28]. Menaka et al. evaluated various deep learning networks for ASD detection, finding customized AlexNet with Linear Frequency Cepstral Coefficients (LFCC) to yield the highest accuracy [29]. Ardakani et al. classified EEG signals from individuals with ASD and healthy individuals using a two-dimensional Deep Convolution Neural Network (2D-DCNN) with data augmentation techniques [30]. The paper explains the proposed work in Section 2, analyzes the results in Section 3, Discussion in Section 4, and concludes the research in Section 5.

3. Proposed work

This study aims to comprehend the impact of mu rhythm on individuals with autism and assess the significance of mu rhythm-based features in subject classification. For this task, three distinct approaches were employed. The first approach involves the utilization of machine learning, where subsets of the entire dataset are extracted using a bandpass filter, and results are evaluated across various models. The second approach utilizes deep learning, employing a CNN-based voting classifier on three images generated from raw EEG data. ICA decomposed images using the SOBI algorithm, spectrograms, and CWT images are obtained, and the results are compared in the voting classifier, which comprises VGG16, InceptionV3, and MobileNetV2. Finally, a hybrid model was introduced, integrating deep learning-based feature extraction followed by machine learning classification, aimed at assessing potential enhancements in the final predictions. Figure 1 gives a detailed elaboration of the complete workflow of the proposed work.

Figure 1.

Block diagram of the proposed work.

3.1. Data acquisition

The intervention protocol was carried out with 10 children with Typical Developing (TD) and 10 children with ASD. The participants’ level of cognition was assessed by the clinician using a DSM V questionnaire before initiating the protocol. The ages of the study participants with ASD and TD ranged from 5 to 7 years. All experimentation was done at the Sri Ramachandra Institute of Higher Education and Research (SRIHER). Participants provided informed consent prior to the commencement of the study, thereby facilitating the collection and utilization of their EEG data for research purposes. The children were presented with videos and audio selected by the clinician. Three datasets are used for this study based on audio stimuli, video stimuli, and audio-video stimuli and the duration of each stimulus is 24 s. Prior to EEG acquisition, participants were instructed to wash their hair, and their scalp was cleaned with spirit to ensure optimal impedance during recording. Ag/AgCl (silver-silver chloride) electrodes were utilized to mitigate potential shifts due to electrode polarization. These electrodes were positioned on the scalp using conductive gel and tapes to enhance conduction. The EEG system utilized for the study was the Nihon Kohden Neurofax MEB9000 version 05-81 with a sensitivity of 7 $\mu$V. Electrode placement followed the 10-20 International Standard, with EEG signals recorded from 19 channels at a sampling frequency of 500 Hz. Electrodes were strategically placed on cortical lobes including frontal (Fp1, Fp2, F3, F7, F4, F8, and Fz), temporal (T3, T5, T4, and T6), occipital (O1 and O2), parietal (P3, P4, and Pz), and central (C3, C4, and Cz). Raw EEG signals underwent filtering with a low-pass and high-pass filter with a cut-off frequency of 0.53 to 70 Hz, and a notch filter was employed to eliminate 50 Hz power line interference noise.

3.2. Data preprocessing

The data is preprocessed using a bandpass filter to isolate alpha and alpha-beta ranges. Subsequently, it is partitioned into 1000 intervals and processed across all channels, focusing solely on the sensorimotor cortex region. After processing, the data is fed into machine learning algorithms for comparative analysis. Before the data is fed into the deep learning model the data are transformed into Spectrograms, Brain Topographical pictures (ICA-SOBI), and CWT pictures. The CNN voting classifier serves as the deep learning model utilized for training these images. The sample images from the dataset are given below in Table 1.

Table 1.

Sample topography and spectrum plots

Plots	Autism spectrum disorder	Typically developing
ICA-SOBI (Brain Topographical Maps)
Spectrogram
CWT images

3.3. Machine learning based approach

For Machine Learning to understand much about Mu rhythm characteristics, highlighting its features is necessary. Taking this into consideration and understanding the previous research works in the domain of mu rhythm, it is understood that the mu rhythm lies in the range of the alpha band, and traces have been found in the beta band as well. Therefore, as the first step, a bandpass filter was applied to highlight the alpha and beta band frequencies (8–13 Hz) and (13–30 Hz), respectively for all channels. This particular task was implemented using a forward-backward filtering approach. This digital filtering function applies the filter twice in a forward and backward direction to minimize phase distortion in the filtered signal, thus resulting in zero-phase filtered signals that align with the original input EEG signal. Auto-Regressive (AR) coefficients, the Higuchi Fractal Dimension, and the Hurst Exponent are time-domain attributes. Discrete wavelet transform, beta power, and alpha power correspond to the frequency domain attributes. Both Time and frequency domain features are extracted for all channels in the alpha and beta frequency ranges. PCA which was used to get around the dimensionality curse, was utilized to identify the significant features. Then, to extract the mu rhythm-related time and frequency domain features, the sensorimotor cortex channels C3, C4, and Cz were picked, and the remaining channels were excluded. Subsequently, these feature vectors were fed into traditional machine learning classifiers like SVM, Random Forest, and Stacked classifiers, which make use of classifiers like KNN, Histogram Gradient Boosting Classifier, and Random Forest.

3.3.1. Feature extraction methods

Auto-Regressive Coefficients: AR coefficients are most commonly used in analyzing time series data due to their ability to capture the underlying dynamics, such as spectral content, frequency band power, and signal coherence. Passing these features instead of the entire signal will significantly reduce the computational complexity of the machine learning model to learn and extract valuable features while training.

$x_{i,c}\left(t\right)={\displaystyle \sum _{i=1}^n}{a}_{i,c}{x}_{i,c}\left(t-i\right)$ (1)

Where $a_i$ be the $i^{\text{th}}$ coefficient of the AR model for channel $c$, and $i=$ 1, $\mathrm{\dots }$, $n$, $n$ is the order of the model. In this work, Burg’s method of order 6 has been utilized for estimating the AR coefficients. This method computes the coefficients by minimizing the sums of squares of forward and backward linear prediction error between the signal and the Auto-regressive model.

Higuchi Fractal Dimension: The Higuchi Fractal Dimension (HFD) is a method for quantifying the fractal complexity of time series data. It offers a means to estimate the fractal dimension of data series without prior knowledge of their underlying dynamics. The algorithm involves partitioning the time series into segments, each of length $k$, and measuring the curve formed by connecting these segments. By plotting the average curve length against the segment length scale ($k$) on a logarithmic scale for different segment lengths. A higher Higuchi Fractal Dimension value indicates increased irregularity or complexity within the time series, implying a more fractal-like structure. The Higuchi Fractal Dimension (HFD) can capture the non-linear features of the signal and give insight into the underlying dynamics of the brain, making it very helpful in EEG feature extraction.

Hurst Exponent: A mathematical method known as the Hurst exponent is used to quantify the long-term memory of time series data. It offers insightful information on the level of resemblance and distant dependencies present in the time series. The Hurst exponent can be calculated in a variety of ways, including Wavelet-based techniques. However, the Rescaled range (R/S) analysis has been used in this research work to implement the Hurst exponent. By breaking the time series into smaller, equal-length pieces (sub-series), this technique determines the range of each segment.

$E\left[{\displaystyle \frac{R\left(n\right)}{S\left(n\right)}}\right]=C\times n^H$ (2)

The Hurst exponent is found using the above power law from equation 2, where $E[x]$ represents the expected value $R(n)/S(n)$ represents the rescaled range, $n$ represents the time of the last observation in the time series data, $R(n)$ represents the range of the data series and $S(n)$ indicates the deviation, to get the exact estimate of the exponent value log must be applied to both sides and this fitted into a straight line, the slope of this line(H) is the estimate of the Hurst exponent value.

Band Power: A bandpass filter was applied to each segment of the EEG data, excluding all frequency bands other than the alpha and beta bands. As the frequency band of filtered data lies within the 8–30 Hz range, three separate bands are obtained whose band powers are: Alpha power, Beta power, and Alpha–beta power. The peak values of each band at specific frequency ranges were recorded, and all of them were determined by applying the power spectral density to the filtered data using the Welch method.

$P_{|{\omega }_1,{\omega }_2|}={\displaystyle \frac{1}{2\pi }}{\displaystyle {\int }_{{\omega }_1}^{{\omega }_2}}\left[S\left(\omega \right)+S\left(-\omega \right)\right]𝑑\omega$ (3)

Equation (3), gives the mathematical interpretation of the power spectral density, where $P_{|{\omega }_1,{\omega }_2|}$ represents the power with the frequency interval, ($\omega$) represents the power spectral density.

Discrete Wavelet Transform: Discrete wavelet transform is a linear feature extraction technique that decomposes the signal into a set of wavelet coefficients, which is achieved through a series of filtering and down-sampling operations. In this work, decomposition level 4 is selected in discrete wavelet transform and applied feature selection technique on the resulting wavelet coefficients. Principal Component analysis is the most commonly used feature selection technique to extract significant information by reducing its dimension.

In this work, 50 wavelet coefficients were extracted from a pool of 1000 coefficients using principal component analysis.

$W_{\phi }(j_0,k)={\displaystyle \frac{1}{\sqrt{M}}}\sum f\left(x\right)\phi j_{0,k}\left(x\right)$ (4)

From Eq. (4), Where $f(x)$, $\phi j0$, ($x$) are the functions of the discrete variables $x=$ 0, 1, 2 $\mathrm{\dots }$, $M-1$. where $W$ returns the wavelet coefficient values.

3.3.2. Machine learning classifiers

Stacking Classifier: Stacking is one of the ensemble learning techniques that has been used to boost the accuracy and the metrics of the model. It operates via a process known as meta-learning. Where there are multiple base models that are trained on the input data and the probabilities or the output values from these base models are given as input to the estimator model and it gives the final predicted output. In this proposed method three models are used as the base ie: Histogram Gradient Boosting Classifier, Random Forest Classifier, and K Nearest Neighbors, and the estimator model used here are Gradient Boosting Classifier as shown in Fig. 2.

Figure 2.

Depiction of stacked classifier model (meta-learning).

K Nearest Neighbor: The K Nearest Neighbor is based on similarity, in the sense this model doesn’t learn anything during the training phase but rather just stores the dataset, and during the classification when a new data point is given to classify it looks for the similar data points and classifies based on that, and this similarity is calculated based on the Euclidean distance between the data points.

Random Forest: A collection of Decision Trees joined together to form a Random Forest. The decision trees are constructed or trained using a subset of the data or feature set. After each decision tree is trained, it produces an output, and the ultimate result from the random forest is the average or the sum of all the decision trees.

Histogram Gradient Boosting Classifier: Gradient Boosting Algorithm is a variety of ensemble learning the way this algorithm works is by building models sequentially and the model upcoming in the next would try to reduce the error of the previous models. The Histogram-based Gradient Boosting Classifier utilizes binning to reduce the feature space size, resulting in faster computation time. Binning involves segregating data into bins and counting occurrence frequencies. Converting numerical data into histograms and combining them with the Gradient Boosting Classifier produces a boosting classifier that leverages histograms.

Gradient Boosting Classifier: The Gradient Boosting Classifier (GBC) is an ensemble learning technique that constructs a series of decision trees sequentially, aiming to rectify the errors of the previous ones by modeling the residuals. Through the fusion of predictions from numerous weak models, commonly decision trees, GBC can attain notable accuracy in classification endeavors. It adeptly manages both numerical and categorical data, furnishes feature importance assessments, and exhibits resilience against overfitting via its regularization methods. Yet, it remains a powerful and widely used algorithm in classification tasks where accuracy is crucial.

3.4. Deep learning based approach

In this work, CNNs are explored to analyze EEG signal images and classify individuals as Autism or Typically Developing. EEG data is filtered and decomposed, generating Brain Topographical maps using the SOBI algorithm. Additionally, Spectrogram and Continuous Wavelet Transformed images are employed as inputs for the CNN to enable comparison. The proposed CNN model incorporates a voting mechanism, combining well-established architectures like MobileNetV2, VGG16, and InceptionV3. Soft voting is implemented, considering class probabilities and averaging them. The final predicted output is determined based on the highest average probability.

3.4.1. Independent component analysis (ICA)

In the context of EEG data analysis, ICA can be used to separate different types of brain activity from the raw EEG data, such as artifacts, noise, and other types of neural activity. Additionally, ICA is able to identify the spatial distribution of each independent component. For example, mu rhythm is typically observed over the sensorimotor cortex, while alpha rhythm is typically observed over the posterior regions of the brain. SOBI (Second Order Blind Identification) ICA is a signal processing technique that is commonly used for separating sources of independent signals from a mixed signal. The input EEG data in EEGLAB, the signal is decomposed by running the ICA algorithm using SOBI method. The ICA algorithm will decompose the mixed EEG data into a set of independent components, each representing a different type of neural activity or artifact. The ICA incorporates SOBI algorithm decomposes data into independent components, enabling the selection of components exhibiting well defines peaks and spatial topographic maps (create a topographic representation of scalp data in a 2D circular perspective, resembling a view from above the head, by employing interpolation on a detailed Cartesian grid).

3.4.2. Continuous wavelet transform

The CWT is a mathematical tool for analyzing non-stationary signals in the time-frequency domain. It decomposes the signal into wavelets at various scales and computes their inner product with the signal at each time point. By convolving the signal with a wavelet function like the Morlet wavelet, it generates coefficients indicating the similarity between the wavelet and the signal at different scales and time points. The CWT calculates the inner product between the signal and a scaled, translated wavelet function at each time point.

3.4.3. Spectrogram

A spectrogram is the visual representation of a signal’s frequency continents over time. The 2D spectrogram is one such method that has the frequency contents in the y-axis and time represented in the x-axis. The intensity of the frequency contents is represented with the darker color. Short-Time Fourier Transform (STFT) is a common method for obtaining a spectrogram by dividing the signal into small segments with some overlapping by applying Fourier Transform on each segment.

3.4.4. Deep learning model

The overall flow of the proposed CNN-based voting classifier is shown below in Fig. 3.

Figure 3.

Overall architecture of the proposed voting classifier.

With these three models, the voting classifier was built; the architectural framework of each model begins with the input data, consisting of SOBI-ICA images, CWT, and Spectrogram images are trained individually in these three respective models. The global average pooling layer is used to reduce the size of the dimension and pass it on to the classification layer. A sigmoid layer activation is then used for binary classification, and the predicted probabilities are then given to soft voting, which averages out all the predictions and selects the best prediction for the final predicted output.

3.5. Hybrid model based approach

Machine learning techniques excel in classification tasks by learning patterns and making predictions based on extracted features. However, they struggle to extract high-level features from complex, unstructured data like images or audio. Deep learning, on the other hand, automatically learns hierarchical representations and extracts meaningful features from raw data. Deep neural networks capture intricate patterns that traditional machine learning models struggle with. However, deep learning models require large labeled datasets and extensive computational resources. To leverage the strengths of both approaches, a hybrid methodology is proposed. It combines deep learning-based feature extraction, utilizing models like CNNs or Recurrent Neural Network (RNN), with machine learning classification. These models are trained on a large dataset to automatically extract relevant features. The extracted features capture crucial underlying patterns. Traditional machine learning algorithms like SVMs or random forests are then employed to build classification models based on these features. The hybrid model aims to enhance prediction accuracy and improve overall metrics by combining deep learning-based feature extraction with machine learning classification. In the frequency range of 8–30 Hz, which encompasses the mu rhythm, spectrograms and CWT images from the C3, C4, and Cz regions are used for feature extraction. VGG-16 is employed as the feature extractor, processing images from both TD and ASD classes. The workflow of this is shown below in Fig. 4.

Figure 4.

Block diagram of proposed hybrid model.

4. Results and discussion

This section presented a comprehensive analysis and discussion of the machine learning based approach, deep learning based approach and hybrid model based approach across all channel and sensorimotor cortex region

4.1. Experiment setup

The experiment setup comprises two main components: an EEG recording device for data collection and training of machine learning and deep learning models. The models were trained on an Intel Core i7 11th Generation processor with 16 GB of RAM, supplemented by a GEFORCE RTX 3050 graphics card for accelerated training.

4.2. Machine learning based analysis

The proposed work in machine learning experimented primarily with four models: logistic regression, random forest, support vector machine, and stacking classifier. The results presented below pertain specifically to the mu rhythm region, which includes data from the C3, C4, and Cz regions in both the alpha and beta bands. It is important to note that the desynchronization of mu rhythm is well-known in this region. A comprehensive analysis of different combinations of region and frequency bands in datasets is provided in the performance analysis section. The classification report for each of the three models is displayed below, where Class 0 represents the autism spectrum disorder class and Class 1 represents the typically developing class. Table 2. Depicts the comparison of ML models.

Table 2.

Comparative analysis of machine learning models

Model	Region and frequency band	Precision	Re-call	F1-score	Accuracy
Logistic regression	All channel – Alpha band	0.48	0.48	0.48	0.48
	Sensory motor cortex – Alpha band	0.59	0.60	0.57	0.58
	All channels – Alpha Beta band	0.68	0.69	0.67	0.68
	Sensory motor cortex – Alpha Beta band	0.71	0.71	0.70	0.71
SVM with polynomial kernel	All channel – Alpha band	0.53	0.56	0.46	0.53
	Sensory motor cortex – Alpha band	0.55	0.68	0.44	0.52
	All channels – Alpha Beta band	0.66	0.71	0.64	0.67
	Sensory motor cortex – Alpha Beta band	0.53	0.63	0.41	0.53
SVM with RBF kernel	All channel – Alpha band	0.58	0.58	0.57	0.58
	Sensory motor cortex – Alpha band	0.73	0.74	0.72	0.72
	All channels – Alpha Beta band	0.68	0.70	0.67	0.68
	Sensory motor cortex – Alpha Beta band	0.58	0.58	0.57	0.58
SVM with linear kernel	All channel – Alpha band	0.50	0.50	0.45	0.49
	Sensory motor cortex – Alpha band	0.59	0.60	0.57	0.58
	All channels – Alpha Beta band	0.69	0.71	0.68	0.69
	Sensory motor cortex – Alpha Beta band	0.71	0.72	0.70	0.71
Random forest	All channels – Alpha band	0.62	0.62	0.62	0.62
	Sensory motor cortex – Alpha band	0.73	0.73	0.72	0.72
	All channels – Alpha Beta band	0.72	0.73	0.72	0.73
	Sensory motor cortex – Alpha Beta band	0.74	0.76	0.74	0.74
Stacking classifier	All channels – Alpha band	0.65	0.65	0.65	0.65
	Sensory motor cortex – Alpha band	0.68	0.68	0.67	0.67
	All channels – Alpha Beta band	0.75	0.75	0.75	0.75
	Sensory motor cortex – Alpha Beta band	0.78	0.79	0.78	0.78

This work compares the performance of four machine learning algorithms on four different combinations of region and frequency bands in the dataset to evaluate their capabilities. Logistic Regression, Random Forest, Support Vector Machine, and the stacking classifier are utilized, each applied to four different combinations of region and frequency bands in datasets such as all channels with alpha Beta band and only alpha band, sensory-motor cortex region channel with alpha Beta band and only alpha band. In the experiments, a stacked classifier was implemented with three base classifiers: a random forest, a KNN, and a Histogram gradient boosting classifier. Subsequently, the meta-classifier was trained using a gradient boosting algorithm to efficiently make the predictions of the base classifiers. The stacking classifier achieved the highest accuracy of 78%, with recall and F1-score exhibiting an increasing trend and reaching 79% and 78%, respectively, for the Alpha Beta band in the sensory-motor cortex region.

Table 3, continued
Plots	Model	Training accuracy	Training loss
	M3
CWT	M1
	M2
	M3

4.3. Deep learning based analysis

In the deep learning aspect, a CNN-based voting classifier is employed on three different images such as SOBI images, spectrogram images, and CWT images. The distribution of training, validation and testing data has 60:20:20. Among all 21 channels, the SOBI images showed the best performance; this subject will be covered in more detail in the performance analysis section that follows. Another method involved retraining the model only with images from the sensory-motor brain region, while still employing the CWT and spectrogram images. The purpose was to compare the performance of the normal deep learning methodology with the dataset containing all 21 channels. However, the SOBI dataset could not be used for this approach due to the manual selection of the sensory-motor cortex region, which would result in dataset imbalance. Augmentation techniques were not suitable in this case because they might alter the structure and important features of the brain maps, limiting their applicability. Here label 1 stands for autism spectrum disorder class and label 0 stands for TD class.

Table 4, continued
Plots	Model	Training accuracy	Training loss
	M3

Table 3 displays the training loss and accuracy curves for all 21 channels across three models (MobileNetV2, VGG16, Inception v3) utilizing SOBI, spectrogram, and CWT images as inputs. Table 4 presents the training accuracy and loss for spectrogram and CWT data across the three models, with an emphasis on the SMC region. Table 5 shows improved accuracies when a voting classifier is combined with spectrogram and CWT approaches. When compared to the Spectrogram, CWT characteristics perform somewhat better, especially in the sensory-motor brain area. Three methods for transforming one-dimensional data into two-dimensional images are presented in Table 6. Unlike traditional Spectrogram or Scalogram methods, the new SOBI method creates brain maps. The SOBI images perform better than the Spectrogram and CWT images in every performance parameter, even though a common classifier is used for all three images. Table 6 shows lower accuracy when compared to Table 5 because it removes the alpha band and uses data from all channels. Performance declines since this new technique does not only highlight mu rhythm aspects.

Table 3.

Training accuracy and loss for SOBI, Spectrogram and CWT with Voting Classifier for all 21 channels

Plots	Model	Training accuracy	Training loss
SOBI	M1
	M2
	M3
Spectrogram	M1
	M2

Table 4.

Training accuracy and loss for CWT and spectrogram with voting classifier in SMC region

Plots	Model	Training accuracy	Training loss
CWT	M1
	M2
	M3
Spectrogram	M1
	M2

Table 5.

Classification results for the Spectrogram and CWT images with voting mechanism on Sensory motor cortex channels (Macro average values)

Plots	Model	Precision	Recall	F1-Score	Accuracy
Spectrogram	Voting	0.66	0.68	0.66	0.67
CWT	Mechanism	0.69	0.69	0.68	0.68

Table 6.

Classification results for the Spectrogram, CWT images, and topographical brain maps with voting mechanism on all channels (Macro average values)

Plots	Model	Precision	Recall	F1-Score	Accuracy
Spectrogram	Voting	0.55	0.56	0.55	0.55
CWT	Mechanism	0.57	0.63	0.52	0.57
SOBI-ICA		0.75	0.75	0.74	0.75

4.4. Hybrid model based analysis

In the hybrid model, VGG16 is employed for feature extraction from the CWT and spectrogram images specifically in the sensory-motor cortex region. VGG16 is chosen over MobileNetV2 and InceptionV3 due to the better accuracy and loss observed in VGG16 from the previous results. The features extracted using VGG16 are combined with the features extracted through machine learning classification. The following results showcase the final classification outcomes achieved by deep learning features along with ML features and machine learning classification models.

Table 7 presents the performance metrics for 3 different machine learning models namely: SVM with polynomial and RBF kernel, Random Forest, and the Stacking classifier. Among all the models stacking classifier performed the best accuracy, precision, recall, and F1-score up to 94%.

Table 7.

Classification results for the Hybrid model on the sensory motor cortex channels

Model	Precision	Recall	F1-Score	Accuracy
SVM – Polynomial	0.93	0.94	0.93	0.93
SVM – RBF	0.89	0.89	0.89	0.89
Random forest	0.81	0.82	0.82	0.82
Stacking classifier	0.94	0.94	0.94	0.94

4.5. Entropy feature analysis

In machine learning-based analysis, using Auto-Regressive (AR) coefficients, the Higuchi Fractal Dimension, the Hurst Exponent, and Discrete Wavelet Transform features from the sensorimotor cortex region (alpha and beta range) with a stacking classifier achieves an accuracy of 78%. Including entropy and fuzzy entropy features improves the accuracy to 81.4%.

In the hybrid model-based analysis, VGG16 is used to extract features from the Continuous Wavelet Transform (CWT) spectrogram. These features are then combined with AR coefficients, the Higuchi Fractal Dimension, the Hurst Exponent, and Discrete Wavelet Transform features, achieving an accuracy of 94% with stacking classifiers. Incorporating entropy and fuzzy entropy features further increases the accuracy to 96.9%. Table 8 depicts the classification results of the machine learning-based analysis and hybrid model-based analysis with entropy and fuzzy entropy features. Table 9. depicts the comparison of existing studies with the proposed approach.

Table 8.

Classification result of the machine learning based analysis and hybrid model based analysis with entropy and fuzzy entropy features

Analysis	Classifier	Accuracy	Precision	Recall	F1-score
Machine learning based analysis	Stacking claasifier	0.81	0.82	0.80	0.81
Hybrid model based analysis		0.96	0.97	0.96	0.96

Table 9.

Comparison of existing study with proposed approach

Authors	Modalities	Features	Classifier	Accuracy
Radhakrishnan et al. [27]	EEG	Spectrogram	ResNet50	81%
Menaka et al. [29]	EEG	Linear frequency cepstral coefficients plot,	Customized AlexNet	90%
Subudhi et al. [18]	EEG	Independent Component Analysis (ICA), non-linear features	Support Vector Machine	90.41%
Abdolzadegan et al. [17]	EEG	Wavelet Transform, Power Spectrum, Fractal Dimension, Fast Fourier Transform (FFT), Lyapunov Exponent, Correlation Dimension, Detrended Fluctuation Analysis, Entropy and Synchronization Likelihood	Support Vector Machines (SVM)	90.57%
Sinha et al. [14]	EEG	Mean, variance, standard deviation, kurtosis, skewness, and Shannon entropy	K-NN classifier	92.8%
Haputhanthri et al. [16]	EEG	Discrete wavelet transform (DWT) technique, mean and standard deviation feature and correlation based feature selection	Random forest classifer	93%
Proposed approach	EEG	Continuous wavelet Transform (CWT), spectrogram, Auto-Regressive (AR) coefficients, Higuchi Fractal Dimension, Hurst Exponent, Discrete Wavelet Transform, entropy and fuzzy entropy	Hybrid model (VGG16 and stacking classifier)	96.9%

5. Conclusion

Autism, a neurological and developmental disorder, is characterized by challenges in communication, social interaction, and repetitive behaviors. Questionnaire-based methods for autism diagnosis present a challenge for researchers because they require careful interpretation and validation. There is a need for objective methods in autism diagnosis to improve accuracy and reliability. This work proposed the hybrid model for autism classification based on analysis of the machine learning-based approach and deep learning-based approach. Conclusively, stacking classifiers with feature extraction achieved the highest accuracy of 81.4% in the Machine Learning approach, while in the Deep Learning approach, the SOBI dataset outperformed spectrogram and CWT images with 75% accuracy using the voting classifier. Incorporating DL-based feature extraction on Spectrogram and CWT, along with ML features from the SMC region, yielded the highest accuracy of 96.9% in ML classification. The hybrid model lies in its potential to improve ASD diagnosis and ultimately enhance the quality of life for individuals affected by disorder. The limitation of the study is predominantly emphasizes the alpha-beta frequency range, possibly overlooking other pertinent frequency bands that might hold significance for the analysis. Future work involves comparing different models and studying their accuracies in ML, exploring additional feature extraction possibilities, and utilizing the new deep learning methods such as transformers, attention learning etc.

Author contributions

Conceptualization – Menaka R; Methodology – Menaka R, Karthik R, Daehan Won; Software, Validation, Visualization – Gaurav Prasanna, Vignesh S, Surya Y; Supervision – Menaka R, Karthik R, Data curation, Formal analysis - Saranya S; Writing – original draft – Gaurav Prasanna, Vignesh S, Surya Y, Writing – review & editing – Menaka R, Karthik R, Daehan Won, Saranya S.

Funding

The authors report no funding.

Conflict of interest

The authors declare that they have no conflicts of interest to report regarding the present study.