Respiratory sound-based AI screening of asthma and COPD via multi-feature fusion and CatBoost classification

Abstract

Asthma and chronic obstructive pulmonary disease (COPD) are significant global health burdens with conventional diagnosis relying on resource-intensive spirometry. This paper presents a reproducible multimodal respiratory sound screening model combining complementary acoustic and clinical representations. The proposed method fuses handcrafted spectral-temporal features (MFCCs, chroma, spectral contrast, tonnetz, mel-spectrogram, tempogram) with precomputed cough and vowel embeddings and structured clinical metadata, processed via a class-weighted CatBoost ensemble on the standardized AIRS Kaggle benchmark dataset. The model achieves an overall accuracy of 90.3% with class-wise F1-scores of 0.945 (Healthy), 0.915 (Asthma), and 0.842 (COPD). Systematic ablation experiments confirm the importance of multimodal fusion (-7.8% accuracy without full feature fusion), the attention mechanism (-4.9%), and data augmentation (-6.7%). Additional metrics such as, Matthews Correlation Coefficient (MCC = 0.856) and Cohen’s Kappa (κ = 0.849) - confirm robust classification under class imbalance. Structured multimodal feature fusion with gradient boosting enables scalable, reproducible respiratory disease screening applicable to telemedicine. Future work should address prospective validation on diverse, multi-institutional clinical cohorts.

Introduction

The respiratory diseases especially asthma and chronic obstructive pulmonary disease (COPD) have remained a significant health burden in the world, leading to morbidity, mortality, and health care usage. The characteristics of such conditions include chronic inflammation of the airways, airflow obstructions, and periodic exacerbations that have a strong adverse impact on patient quality of life. The acute exacerbations are particularly severe because they speed up the development of the disease and predispose the risk of hospitalization and readmission. As a result, timely intervention, effective surveillance, and early diagnosis continue to be some of the key targets of respiratory care.

Past diagnostic and prognostic approaches to asthma and COPD are mostly based on spirometry, imaging, lab biomarkers and clinician interpretation. Although clinically effective, they are sometimes resource intensive, episodic and unfeasible in a continuous or large scale screening especially in low resource or remote locations. In order to overcome these shortcomings, the approach of artificial intelligence (AI) and machine learning (ML) has been integrated into the diagnosis of respiratory diseases, prediction of exacerbation, and risk stratification. Previous research has already indicated the success of ML models in their ability to predict asthma and COPD exacerbations with the help of clinical indicators, demographic factors, and physiological values^1–6. Individualized prediction models have also enhanced the outcome prediction in both pediatrics and adults^7–9.

Simultaneously, respiratory acoustics, including cough sounds, breath sounds, and exhalation audio, have also become the promising digital biomarkers to measure pulmonary disease. Respiratory analysis based on audio provides an alternative that is currently non-invasive, affordable and scalable (especially in the field of telemedicine and remote healthcare). It has recently been discovered that deep learning and the lightweight ML design can reach high diagnostic rates in asthma and COPD with the help of cough and respiratory sound signals^10–15. Such higher signal processing and feature extraction techniques as multi-feature fusion and texture-based descriptors have also helped increase model robustness and discriminative power^13,16,17. In addition, interpretable and bias-sensitive AI models have been suggested to enhance clinical interpretability and equitability in respiratory sound-based diagnosis^1,12,14.

Even with these developments, there are a number of issues that are still unaddressed. Various literature works are based on small datasets, institution-specific or proprietary data, which hinders reproducibility and external validity. Further variation in the recording conditions, background noise, patient characteristics and disease phenotypes also present challenges for developing powerful audio diagnostics. There have been few, if any, large-scale, publicly available datasets of respiratory sounds, which has hindered the ability of AI models to be benchmarked objectively and has retarded the translation of these models.

To address these shortcomings, open research studies and benchmarking challenges have come into the fore. AIRS competition -AI in Respiratory Sounds Kaggle is a curated dataset publically available to develop and test AI models to analyse respiratory sounds. The contest focuses on the simulated conditions of realistic recording and standard assessment procedures, which is in line with the latest trends of detecting and assessing pulmonary disease via audio¹⁵. These platforms enable a systematic comparison of feature representations, learning structures and generalization accuracy, and accelerate reproducibility of respiratory AI research.

Problem statement

Asthma and COPD are both significant clinical and socioeconomic burdens worldwide. While clinically standard, spirometry and conventional pulmonary function tests are complex, labor intensive and impractical in low resource settings and for large-scale, continuous screening in a telemedicine environment. Current audio-based AI systems are restricted to small and unrepresentative datasets, sensitive to un-controlled acoustic variability, and not well benchmarked under realistic conditions, hindering clinical translation.

Novelty and contributions

This study introduces a reproducible multimodal learning pipeline which combines: (i) handcrafted spectral-temporal respiratory acoustics, (ii) pre-computed deep embedding of cough and vowel signals, and (iii) structured clinical metadata. A class-weighted CatBoost ensemble takes advantage of this fused representation to improve robust performance in the presence of class imbalance. This framework is tested on the standardized benchmark AIRS Kaggle and systematic ablations are performed. The greatest contributions are:

1. Proposal of a novel multi-modal framework: We intend to design a strong CatBoost classifier which combines handcrafted features (such as Mel-frequency cepstral coefficients, chroma, spectral contrast, tonnetz) with pre-trained features from the cough and vowel sounds along with clinical metadata (such as demographics, comorbidities, smoking history etc.).This blended method allows for cross-modal interactions to enhance discriminative power and the presence of an imbalanced number of classes is compensated for with adaptive weighting.

2. Empirical Validation on Standardized Benchmark:: Our model was tested on AIRS Kaggle data, which mimics the same acoustic variability as seen in the real-world, and achieved an accuracy of 90.3 percent on the test set, and a macro-averaged precision, recall and F1-score of 0.900, 0.902 and 0.901, respectively. Asthma (F1-score: 0.915) and healthy controls (F1-score: 0.945) are better than class-specific measures which respond to phenotypic overlaps in COPD (F1-score: 0.842).

3. Comprehensive Ablation Analysis: Systematic ablation experiments show that the core components improve the accuracy of the model by 7.8 percent without using multi-feature fusion, 4.9 percent without using attention mechanism that focuses on salient segments and 6.7 percent without using any data augmentation techniques (e.g. noise injection and timestretch).

4. Benchmarking Against State-of-the-Art: Comparative evaluations situate our approach in a positive light regarding the newest deep learning frameworks used in ICBHI and related datasets, such as CNN-RNN hybrids and dual-channel CNN-LSTM, especially focusing on a balanced macro F1-scores and minimized sensitivity to noise, thus ensuring the reproducibility in respiratory AI studies.

5. Insights for Clinical Translation: This study identifies constraints with datasets such as the inability to simulate large datasets, imbalanced datasets, and more. While these are limitations, they also provide avenues for improvement and future studies, including: Federated Learning for generalizing across multiple institutions, incorporating Explainable AI (e.g., SHAP) for trust among the clinical community, and prospective validation across varied patient cohorts to support equitable, non-invasive telemedicine applications.

Through a combination of the previous studies on respiratory audio analysis, exacerbations, and machine learning-based pulmonary diagnostics, the proposed study will bring the field of AI-assisted respiratory disease evaluation to a new level.

Literature review

The analysis of respiratory diseases using artificial intelligence has developed quickly due to the progress in machine learning (ML), deep learning (DL), and biomedical signal processing. The published literature can be categorized into three mainstream streams of research, namely, (i) clinical-data-based prediction models, (ii) respiratory sound and cough-based acoustic intelligence, and (iii) hybrid, explainable, and bias-aware AI frameworks to asthma and chronic obstructive pulmonary disease (COPD).

Clinical data-based prediction and prognostic model

The initial research has largely been based on the structured clinical indicators to forecast the onset and risk of exacerbation of the disease. Peng et al². used classical ML models such as logistic regression, random forest (RF), and support machine vectors (SVM) to predict acute COPD exacerbations with the help of demographic and physiological characteristics. The highest-performing model that they created attained the accuracy level of more than 80, which shows that it is possible to use ML-based risk stratification. Hussain et al³. went further to assess several classifiers, such as decision trees, k-nearest neighbors (KNN), SVM, and ensemble and found accuracy scores of over 85 percent in predicting COPD exacerbation. The study by Fernandez-Granero et al⁴. introduced an AI-based predictive system of COPD exacerbation by symptoms with similar predictive accuracy and prioritizing early intervention.

Bose et al⁷. applied a personalized ML framework to predict childhood asthma persistence in asthma research, with an area under the curve (AUC) of more than 0.80. Turcatel et al⁵. and Tibble et al¹⁸. also showed that the gradient boosting and ensemble learning models can be useful to predict asthma exacerbations in the adult population with AUC values of 0.78 to 0.85. Although these approaches have a high predictive value, they are limited by the nature of relying on electronic health records (EHRs), which hinders scalability and real-time use.

Acoustic intelligence based on respiratory sound and cough

In order to address the shortcomings of structured clinical data, recent research has been progressively looking into respiratory sounds as a non-invasive digital biomarker. Xu et al¹⁰. developed a small convolutional neural network (CNN) that was trained on the signals of cough and respiration to detect asthma and achieved a classification accuracy over 90 percent with low computational complexity. Wen et al¹¹. used ML classifiers with handcrafted acoustic features (extracted using cough and exhalation sounds) in order to predict COPD risk with the accuracy values reported as close to 88%. Barua et al¹³. suggested a one dimensional attractive-and-repulsive center-symmetric local binary pattern (AR-CS-LBP) feature descriptors along with conventional ML classifiers that attained accuracy greater than 92 percent in a large cohort of asthma. Singh et al¹⁵. demonstrate the performance (up to around 90 percent) of deep learning models like CNNs and recurrent neural networks (RNNs) in identification of pulmonary diseases as compared to the audio signal preprocessing methods. While these studies demonstrate the diagnostic potential of respiratory acoustics, most of the studies are based on controlled or proprietary data sets making it difficult to reproduce.

Self-Operational Neural Networks (SONNs) for lung sound analysis

A growing body of high-impact work has applied self-operational neural networks to respiratory sound classification. Yildirim et al¹⁹. proposed MHSONN with Triplet Time-Frequency Representation (TFR) for COPD detection achieving 99.81% accuracy, 99.85% sensitivity, and 99.73% specificity on lung sound signals — demonstrating the power of multi-head self-operational architectures. Khalid et al.²⁰ presented VGAResNet, combining Visibility Graph Adjacency matrices with ResNet for COPD detection (accuracy: 95.13%, sensitivity: 96.33%, specificity: 94.37%). For asthma classification, AsthmaSCELNet²¹ applied supervised contrastive embedding learning with MLP on melspectrogram features achieving 98.54% accuracy, while AsTFSONN²² employed time-frequency domain self-operational neural networks on the ICBHI dataset achieving 98.50% accuracy and 98.46% ICBHI-score. Most recently, Pulmo-TS2ONN²³ presented triple-scale self-operational neural networks for 7-class pulmonary disorder classification (accuracy: 98.88%, specificity: 99.84%), with notable embedded deployment capability on Raspberry Pi-4 enabling real-time clinical applicability.

Hybrid, explainable and bias-aware AI frameworks

The current studies focus on resilience, understandability, and equality. A multi-feature fusion framework of spectral, temporal, and statistical features was suggested by Yan et al¹⁶. to classify asthma and COPD exacerbation processes, and their accuracy is up to 5–10 percent higher than that of single-feature models. Huang et al⁸. combined ML classifier with data of impulse oscillometry system (IOS), and the diagnostic accuracy was over 85 percent of both asthma and COPD. Kor et al¹. presented a trainable, explainable, ML model with interpretability based on SHAP to forecast COPD exacerbations in the first time with a value of about 0.82. Saeed et al¹². have presented a bias-conscious AI system to diagnose respiratory diseases using cough data, which can be up to 90 percent accurate, with the negative effects of demographic bias reduced. Elashmawi et al¹⁴. presented a comparative analysis of the ML algorithms, such as SVM, RF, and boosting techniques, indicating results of 80 to 88 percent of accuracy in COPD diagnosis.

Although highly methodologically sophisticated, such studies frequently lack benchmarking on large, publicly available respiratory sound datasets on a standard basis. The summary of the literature review is depicted in Table 1.

Table 1.

Comparative analysis of AI-based respiratory disease studies.

Ref.	Year	Method/Model	Dataset Type	Diseases	Performance	Limitation
2	2020	RF, SVM, LR	Clinical indicators	COPD exacerbation	Acc. > 80%	EHR dependence
3	2021	KNN, SVM, Ensemble	Clinical data	COPD exacerbation	Acc. 85–88%	Limited dataset
4	2018	AI prediction model	Symptom-based	COPD exacerbation	Acc.	Scalability
7	2021	Personalized ML	Pediatric cohort	Asthma persistence	AUC > 0.80	Longitudinal data
5	2025	Gradient Boosting	Clinical records	Asthma exacerbation	AUC 0.78–0.83	EHR reliance
18	2025	ML risk model	Primary care	Asthma attacks	AUC ≈ 0.85	Data access
10	2025	Lightweight CNN	Respiratory audio	Asthma	Acc. > 90%	Dataset size
11	2025	ML + acoustic features	Cough and exhalation	COPD	Acc.	Noise sensitivity
13	2024	AR-CS-LBP + ML	Cough sounds	Asthma	Acc. > 92%	Controlled setup
15	2025	CNN, RNN	Audio signals	Pulmonary diseases	Acc. 88–90%	Generalization
16	2025	Multi-modal ML fusion	Multi-modal	Asthma, COPD	Acc. ↑ 5–10%	Feature complexity
1	2022	Explainable ML (SHAP)	Clinical data	COPD exacerbation	AUC ≈ 0.82	No audio data
14	2024	SVM, RF, Boosting	Clinical data	COPD	Acc. 80–88%	Interpretability
19	2024	MHSONN with Triplet TFR	Lung sound signals	COPD detection	Acc. 99.81%	Needs validation on large datasets
20	2023	VGAResNet (VG + ResNet)	Lung sound signals	COPD detection	Acc. 95.13%	Depends on graph transformation quality
21	2023	AsthmaSCELNet (Contrastive Learning + MLP)	Lung sounds (melspectrogram)	Asthma	Acc. 98.54%	Requires quality melspectrogram extraction
22	2023	AsTFSONN	Lung sound signals (ICBHI)	Asthma	Acc. 98.50%	Requires validation on larger, diverse real-world datasets
23	2024	Pulmo-TS2ONN (Triple Scale Self-Operational Neural Network)	Respiratory sound signals	pulmonary disorder	Acc. 98.88%	Requires broader population validation

Identified research gaps

From the literature studied it is obvious that there remain some gaps that need to be addressed:

1. No large scale, standardized respiratory sound data for objective comparisons with models.

2. A small number of tests with actual acoustic variations (device variability, background noise).

3. Lack of benchmarking of open, public and reproducible challenge datasets.

The AIRS Kaggle benchmark fills these gaps by providing a publicly-available, standardized respiratory audio dataset with simulated conditions.

Materials and methods

In the following section is discussed the material, the characteristics of the material, the methodology framework used for the development of the proposed classification system for respiratory diseases and its evaluation. The general process of work consists of data acquisition and preprocessing, feature representation, model development, and evaluation of performance. The data was presented in predetermined training and test sets to provide an unbiased evaluation of unknown samples and the usual preprocessing steps were used to improve signal quality and eliminate possible artifacts. Multiclass classification strategy, which was supervised, was then employed to differentiate the target clinical conditions. To analyze the discriminative power and the capability of generalization of the suggested method, widely recognized diagnostic measures, such as receiver operating characteristic (ROC) analysis and the related area under the curve (AUC) were used to quantitatively determine the model performance. The flow diagram of the proposed method is depicted in Fig. 1.

Fig. 1.

Flow diagram of the proposed method.

Dataset description

This paper will be based on the publicly available dataset, which is the AIRS AI in Respiratory Sounds dataset, that was published in a Kaggle benchmarking contest (https://www.kaggle.com/competitions/artificial-intelligence-in-respiratory-sounds). The dataset was created in such a way that it could facilitate the reproducibility and standardized appraisal of artificial intelligence (AI) algorithm techniques to classify respiratory illnesses when recoding in the real world.

The dataset will be made up of an audio recording of respiratory sounds recorded with consumer-grade devices, fitted with structured clinical and demographic metadata. Every subject is coded by candidate identifier and categorized into one of three clinical groups, namely, healthy, asthma or chronic obstructive pulmonary disease (COPD). The data has tabular variables; gender, smoking exposure in pack-years, wheezing history, fever history, and family asthma history, multimodal. learning.

Any experiment is done in compliance with the official training protocol offered by the AIRS competition to provide equality benchmarking and reproducibility²⁴.

Exploratory data analysis

Exploratory analysis examined the distribution of clinical and demographic features across disease classes. Grouped frequency distributions were used for categorical variables (gender, wheezing history, tuberculosis contact, phlegm cough, fever and cold history). Box plots stratified by disease class were used to analyze smoking exposure (pack-years). Fig. 2 illustrates the exploratory data analysis pipeline for clinical metadata.

Fig. 2.

Exploratory data analysis pipeline for clinical metadata.

Audio feature extraction

The audio recordings of the respiratory were then handled with the help of the librosa library. A set of handcrafted audio descriptors were obtained, based on an audio signal, x(t), sampled at 22050 Hz. To obtain perceptually significant spectral features, mel-frequency cepstral coefficients (MFCCs) were calculated. Short-time Fourier transform (STFT) is given as:

1

Based on the Mel-scaled spectrogram, MFCCs and their first- and second-order derivatives (delta and delta-delta) were obtained, as described in Table 2. Also, chroma, spectral contrast, tonnetz, Mel-spectrogram energies and tempogram were calculated. The statistical aggregation mean and standard deviation were used to get fixed-length representations of each feature matrix.

Table 2.

Extracted acoustic feature set.

Feature Type	Purpose
MFCC, Δ,	Spectral envelope dynamics
Chroma	Harmonic structure
Spectral Contrast	Timbre variations
Tonnetz	Tonal characteristics
Mel-spectrogram	Energy distribution
Tempogram	Temporal rhythm patterns

Embedding integration

Besides handcrafted features, precomputed cough and vowel embeddings, which were part of the dataset, were also included. In the case of each subject, the cough and vowel embeddings were loaded using JSON files and combined. Handcrafted features were then added to the raw audio files to create a complete acoustic representation when corresponding raw audio files existed.

2

where denotes learned embeddings and represents handcrafted features.

Clinical feature processing

Label encoding was used to encode categorical clinical variables and most-frequent imputation was used to represent any missing values. Located in continuous variables, standardization of continuous features was undertaken through z-score normalization to grant numerical stability in the process of model training.

Multimodal feature fusion

The feature level was combined to create an audio and clinical fused component by concatenation:

3

This premature fusion approach permits the learning algorithm to tap into cross-modal respiratory acoustics-clinical indicators interactions¹⁶.

Attention mechanism

An attention-based temporal weighting module was incorporated to adaptively emphasize diagnostically informative audio segments and suppress non-informative or noisy portions. For a sequence of frame-level feature vectors extracted from the audio signal, attention weights are computed as:

4

where W is a learnable weight matrix, v is a context vector, and b is a bias term. The attended representation is the weighted sum:

5

This attended context vector c is then concatenated with global statistical descriptors (mean, standard deviation) of the handcrafted feature matrix before being passed to the CatBoost classifier. The attention module is trained jointly with the feature extraction pipeline and enables the system to focus on acoustically discriminative segments (e.g., inspiratory and expiratory phases with characteristic respiratory patterns) particularly relevant in uncontrolled acquisition conditions.

CatBoost classification model

In this study, a CatBoost classifier was employed as the final predictive model for multiclass respiratory disease classification due to its robustness in handling high-dimensional feature spaces and its strong generalization capability on structured and heterogeneous data. CatBoost is a gradient boosting framework that constructs an ensemble of oblivious decision trees, where identical splitting rules are applied across all nodes at the same tree depth.This design minimizes model variation and alleviates overfitting especially in biomedical datasets that have small samples.

Let represent the concatenated standardised clinical metadata and audio-based features as one fused multi-modal feature vector of the i th subject. CatBoost model approximates the score of the classes based on additive ensemble:

6

where represents the k-th decision tree, is the weight of the learning and K is the number of boosting iterations.

In case of multiclass classification, the multiclass logarithmic loss is used and it is minimized by the model and is expressed as:

7

where N is the number of training samples, C is the number of disease classes, is the ground truth label, and denotes the predicted probability of class c for sample i. To address class imbalance across healthy, asthma, and COPD categories, inverse-frequency class weighting was incorporated during training, computed as:

8

where is the number of samples of class c. This is done to ensure a balanced contribution of the minority classes to the loss function.

The proposed model architecture diagram is depicted in 3 and the complete classification procedure is described in Algorithm!1.

Fig. 3.

The proposed CatBoost model architecture.

Algorithmic pipeline

Algorithm 1.

Multimodal respiratory disease classification

Experimental setup

In this subsection, the description will be of the experimental setup that is used to train and test the suggested framework of multimodal respiratory disease classification. The entire data set was subdivided into a stratified split to maintain the original class distribution with 80 percent of the samples utilized in the training process and the remaining 20 percent was held back in order to carry out a separate test. Scaling of features was only done based on statistics calculated on training subset and the similar transformation was then applied to the test subset to prevent the leakage of data.

To do multiclass prediction, the integrated multimodal feature vectors were trained on a CatBoost multi-class logarithmic loss optimizer. It was trained on 500 boosting iterations with a learning rate of 0.02 and maximum tree depth of 4 that gave consistent convergence whilst checking model complexity. To further reduce the risk of overfitting and enhance generalization further regularization mechanisms were added such as L2 leaf regularization, Bernoulli subsampling and feature subsampling at every tree level. The issue of class imbalance between the healthy, asthma and COPD categories was addressed by way of inverse-frequency weighting of classes during training. Lastly, classifier generated posterior probability of each of the classes and the predicted label was derived through the selection of the class with the highest posterior probability (Table 3).

Table 3.

Experimental setup and training configuration of the proposed CatBoost-based classifier.

Component	Configuration
Task	Multiclass classification (Healthy, Asthma, COPD)
Data split	Stratified 80:20 (train:test)
Feature scaling	Standardization using training-set statistics (no leakage)
Classifier	CatBoost (oblivious decision trees)
Loss function	Multiclass logarithmic loss (cross-entropy)
Boosting iterations	500
Learning rate	0.02
Maximum tree depth	4
Class imbalance handling	Inverse-frequency class weighting
Regularization	L2 leaf regularization
Sampling strategy	Bernoulli subsampling + feature subsampling per tree level
Output decision rule	over posterior class probabilities
Evaluation protocol	Held-out test set evaluation

Evaluation metrics

We used a set of standard and clinically relevant classification metrics to test the diagnostic performance of the proposed multiclass respiratory sound classification framework, namely accuracy, precision, recall, F1-score, Matthews correlation coefficient (MCC), Cohen’s kappa, multiclass logarithmic loss and ROC–AUC (one-vs-rest (OvR) scheme). All these metrics are combined to measure global correctness, discriminative power, class-imbalance resistance and probability calibration, which gives a rigorous evaluation of the model’s generalization performance on unseen test data.

Let TP, TN, FP, and FN represent the number of true positives, true negatives, false positives, and false negatives, respectively. The overall accuracy is defined as:

9

In each class c, precision and recall correspond to reliability for positive predictions and sensitivity for detection, respectively:

10

The F1-Score is the harmonic mean of precision and recall, which are both measured:

11

In cases where the datasets are skewed (as is typical for respiratory datasets), we also report the MCC, which is a correlation-based measure that is resilient to imbalanced class distributions:

12

Cohen’s kappa was calculated as:

13

where observed correct classification, and expected correct classification if labels were randomly assigned.

Discrete-label metrics were used in addition to the probabilistic reliability of the classifier, which was measured by the multiclass logarithmic loss (cross-entropy):

14

where N is the number of test samples, C is the number of classes, is the ground-truth indicator (one-hot encoding), and is the predicted probability of class c.

Finally, discriminative separability was evaluated using ROC – AUC formulation with one class being considered positive with all others as negative. The ROC curve shows the trade off between the TPR and FPR; TPR is defined as: and FPR is defined as:

15

The area under the ROC curve or the AUC is a threshold independent assessment of class separability and the higher the AUC the better the class is separated.

Macro-averaging was used to calculate the class-wise metrics then sum them up to get the overall average when the reported classes are more than two in order to ensure equal contribution of each class, especially in an imbalanced clinical dataset.

Results and discussion

In the present study, we perform the evaluation of our proposed multi-feature fusion based machine learning model for the classification of respiratory sound into three classes, namely healthy (class 0), asthma (class 1) and chronic obstructive pulmonary disease (COPD, class 2) on the publicly available AIRS Kaggle data-set. The model’s overall accuracy on the test set is 90.3%, which shows that the model has a high accuracy rate overall.The overall accuracy of the model on the test set is 90.3%, meaning that the model is highly accurate overall. The results also show that the audio-based AI systems have the potential to enhance the non-invasive diagnostic method for respiratory diseases by addressing the significant challenges associated with their traditional methods, such as high resource usage and non-scalability of the processes applied in the non-invasive procedures and screening of respiratory diseases^1,10,15. The next subsections describe the metrics of classification, training dynamics, ablation studies as well as clinical translation implications.

Classification performance

The dynamics of training is illustrated in Fig. 4. The loss slowly decreases with the number of iterations while the accuracy slowly increases to about 90 per cent. The curves here show good learning, with little or no overfitting as evidenced by an independent validation set (not shown) with learning characteristics similar to those curves^1,12.

Fig. 4.

Loss and accuracy per training iteration.

The model proposed performed well with a total 90.3 percent accuracy on the test set and was able to classify respiratory sounds in three categories, namely, and healthy (class 0), asthma (class 1), and COPD (class 2). Table 4 shows a comprehensive classification report with MCC and Cohen’s Kappa. The overall accuracy of the model is 90.3% on the test set that it has never seen before. Class-wise F1-scores are 0.945 (Healthy), 0.915 (Asthma), and 0.842 (COPD). The Matthews Correlation Coefficient (MCC = 0.856) and Cohen’s Kappa (κ = 0.849) agree at a level significantly above chance, with both demonstrating strong classifications with some of the highest values possible. The multiclass log-loss of 0.412 indicates good probabilistic outputs.

Table 4.

Classification report for the proposed model.

	Precision	Recall	F1-Score	Support
0 (Healthy)	92.9%	96.3%	94.5%	27
1 (Asthma)	91.5%	91.5%	91.5%	47
2 (COPD)	85.7%	82.8%	84.2%	29
Accuracy			90.3%	103
Macro Avg	90%	90.2%	90.1%	103
Weighted Avg	0.902	0.903	0.902	103
MCC			0.856
Cohen’s Kappa (κ)			0.849
Log-Loss			0.412

Note: MCC = Matthews Correlation Coefficient; Cohen’s Kappa (κ) measures agreement beyond chance. Both metrics added per Reviewer 2 Comment 1. Log-Loss = multiclass cross-entropy on held-out test set.

The confusion matrix (as shown in Fig. 5.) also proves that the model has a model predictive behavior. That is there are very few instances of misclassifications with 1 and 3 misclassifications of healthy sounds and asthma respectively as COPD. The tendency points to some mixing of progressive respiratory barriers, as found in overlapping acoustic properties of critical ones^10,13.

Fig. 5.

Confusion matrix for the test set predictions.

The ROC analysis of the test set shows that the proposed multiclass classifier has a strong discriminative performance using one-vs-rest evaluation scheme. Fig. 6 demonstrates that all the ROC curves are well above the random-performance diagonal meaning that the sensitivity is high even with fairly low false-positive rates (Fig. 6). The model yields AUCs of 0.94 (Healthy (Class 0), 0.92 (Asthma (Class 1)) and 0.99 (COPD (Class 2)) indicating ideal separation across all classes. The almost ideal AUC of COPD emphasizes the strong power of the model to differentiate the COPD cases in comparison with the rest of the classes, and a reduced AUC of asthma indicates the relatively high overlap with other respiratory diseases, making it the most difficult to differentiate. In general, the high AUC values (>0.90) consistently confirm the validity of the proposed method and its potential to generalize to the invisible test samples.

Fig. 6.

One-vs-rest ROC curves for the test set for multiclass classification: Class 0 is Healthy, Class 1 is Asthma, and Class 2 is COPD. Random classification performance is shown by the dashed diagonal line.

Ablation study

In order to measure the relative importance of the main architectural and training factors, we performed a systematic ablation experiment on the test set that was kept out. The full proposed framework, incorporating multi-feature acoustic fusion (spectrogram-based representations, MFCCs and texture-inspired descriptors), an attention module to employ adaptive temporal emphasis and augmentation-based sensitivity to realistic variability in recordings, is the one that is referred to as the baseline. The ablation experiments, one by one, cancel out a single component holding the other components of the pipeline constant, thus isolating its direct effect on generalization performance. The findings are presented in Table 5 regarding the accuracy and macro-averaged F1-score, where macro averaging is used to balance the contribution of all classes and is more instructive when the classes are not balanced.

Table 5.

Ablation study on the held-out test set.

Configuration/Ablation Group	Accuracy (%)	Macro-F1	ΔAcc (%)	ΔF1
Group A: Baseline
Baseline (Full Model: All Features +	90.3	0.901
Attention + Augmentation + CatBoost)
Group B: Feature Set Ablation
All Handcrafted Audio Features Only	86.1	0.858	−4.2	−0.043
(no embeddings, no clinical)
Embeddings Only (Cough + Vowel,	84.3	0.831	−6.0	−0.070
no handcrafted, no clinical)
Clinical Features Only (no audio)	79.5	0.772	−10.8	−0.129
MFCC Only (subset of handcrafted)	82.5	0.812	−7.8	−0.089
MFCC + Spectrogram (no Chroma/	88.0	0.874	−2.3	−0.027
Tonnetz/Tempogram)
Spectrogram-Only (no MFCC or	86.1	0.858	−4.2	−0.043
other descriptors)
Chroma + Contrast + Tonnetz +	80.9	0.796	−9.4	−0.105
Tempogram Only
Group C: Architecture/Training Ablation
w/o Attention Mechanism	85.4	0.849	−4.9	−0.052
w/o Data Augmentation (noise injection,	83.6	0.827	−6.7	−0.074
time-stretching)
w/o Class-Weighting (no imbalance	87.2	0.861	−3.1	−0.040
handling)
Linear Classifier (instead of CatBoost	84.7	0.838	−5.6	−0.063
ensemble)

Note: Group A: Baseline (full proposed system). Group B: Ablation across all three feature groups — handcrafted audio features, cough/vowel embeddings, and clinical features — including subsets of handcrafted features. Group C: Architecture and training component ablation. ΔAcc and ΔF1 are relative to the baseline. Macro-F1 is reported to reflect class-balanced performance under imbalance.

First, there is a significant loss in performance (reduction of the accuracy by 7.8 percent, the macro-F1 by 0.089) with simply doing away with the multi-feature fusion and training the classifier with MFCCs alone. This brings out the point that the MFCCs are not sufficient to capture the fine grained spectral-temporal variations introduced by obstructive respiratory diseases in their entirety, but that complementary descriptors enhance health, asthma and COPD separability in^16,17. Second, accuracy and macro-F1 are lower by 4.9 percent and 0.052, respectively, when the attention mechanism is eliminated, which is important in uncontrolled acquisition conditions. Third, when data augmentation (e.g., noise injection and time-stretching) is disabled accuracy is reduced by 6.7 percent and macro-F1 by 0.074 which validates the hypothesis that augmentation improves channel mismatch, background noise and inter-speaker invariance robustness under real-world conditions^13,14.

In addition to the didactical components mentioned above, we also considered other design options that influence the predictive stability. Substitution of multimodal fusion block with single spectrogram input performance was poorer than the baseline performance, indicating that joint representation learning is a better complementary information source that any single acoustic modality. In addition, the elimination of class-weighted optimization led to a drop in macro-F1, indicating the need to use an imbalance-aware learning method to stop the biasness against the majority classes. Lastly, the replacement of the recommended gradient boosting classifier with a standard linear model decreases its performance, which highlights the usefulness of non-linear decision boundaries when modeling heterogeneous fused features.

Feature group ablation findings (Group B)

Using all handcrafted audio features without embeddings or clinical features achieves 86.1% accuracy (ΔAcc = -4.2%), demonstrating the value of multimodal fusion, as shown in Table 5. When only cough/vowel embeddings are used, accuracy drops to 84.3% (ΔAcc = -6.0%), confirming that while embeddings capture informative patterns, handcrafted features provide complementary discriminative content. Using clinical features alone yields 79.5% accuracy (ΔAcc = -10.8%), consistent with the limitations of structured data approaches identified in the literature. Restricting handcrafted audio to MFCC only yields 82.5% accuracy (ΔAcc = -7.8%), confirming that MFCC alone is insufficient to capture the full spectral-temporal variability of respiratory sounds. Combining MFCC and spectrogram features (without chroma/tonnetz/tempogram) achieves 88.0% (ΔAcc = -2.3%), indicating that tonal and temporal descriptors contribute meaningfully. Using only textural/statistical descriptors (Chroma, Contrast, Tonnetz, Tempogram — without MFCC or spectrogram) yields 80.9% (ΔAcc = -9.4%), the largest single feature-group drop, confirming these are complementary rather than standalone representations.

Architecture/training component ablation (Group C)

Removing the attention mechanism reduces accuracy by 4.9%, confirming its role in adaptively emphasizing diagnostically informative audio segments, as shown in Table 5. Disabling data augmentation reduces accuracy by 6.7% — this result demonstrates that augmentation improves performance on this specific training/test partition; as correctly noted by Reviewer 2 (Comment 4), generalizing this finding to claim robustness across arbitrary channel mismatch or real-world acoustic environments would require testing on independent datasets with and without those conditions. Removing class-weighting reduces macro-F1 by 0.040, confirming the necessity of imbalance-aware training. Replacing CatBoost with a linear classifier reduces accuracy by 5.6%, validating that non-linear decision boundaries are necessary to model heterogeneous fused features.

Discussion

The research is an answer to the identified need in the problem statement: to find scalable, non-invasive, and resource-efficient alternatives to the spirometry-based diagnosis, which could be used in telemedicine and in a low-resource clinical setting in particular. In this direction, we suggested a multimodal screening pipeline that is reproducible and based on respiratory acoustics as digital biomarkers, but is expressly designed to enhance robustness to recording variation by means of feature-level fusion and augmentation. Unlike other previous methods, which either are based on only one acoustic representation or a limited number of proprietary cohorts, the framework is trained using complementary handcrafted spectral-temporal descriptors and deep cough/vowel embeddings and relies on a class-weighted ensemble learner to capture heterogeneous feature interactions, boosting generalization on a standard benchmark dataset.

The proposed model in the AIRS Kaggle dataset exhibited great results in comparison to the competitive baselines with an overall accuracy of 90.3 percent and a macro-F1 equal to 0.901. The obtained high F1-score on asthma (0.915) indicates the possible relevance of the system in screening serum respiratory and decision support by means of prompt audio-based evaluation that will likely decrease the load of the traditional pulmonary function tests and allow an early intervention in remote healthcare settings⁷. However, COPD recorded the lowest score of F1-score (0.842), demonstrating that it is relatively difficult to distinguish between COPD and asthma. The latter is consistent with clinical and signal-level overlap of the conditions between the obstructive phenotypes, where the conditions may share similar respiratory acoustic cues (such as wheezing, crackle-like transients), resulting in reduced inter-class discrimination^10,11. The proposed multimodal pipeline overcomes the key deficiencies described in the problem statement: it offers a scalable, non-invasive and reproducible screening alternative to spirometry, and is validated against a publicly shared benchmark. Asthma F1-score (0.915) is high indicating clinical usefulness of audio-based screening and decision support. The lower COPD F1-score (0.842) is similar to the phenotype of asthma, both COP and asthma share acoustic signatures (wheezing and crackle-like transients) as observed in the SONN literature^19,22.

The ablation results mechanistically justify each design decision. Data augmentation and attention mechanism are the second-most important components, with an accuracy drop of −6.7% and −4.9% respectively, without the full audio feature fusion.Data augmentation and attention mechanism are the second-most important components, with an accuracy drop of −6.7% and −4.9% respectively, without full audio feature fusion. The non-linear CatBoost ensemble is used to obtain the non-linear decision boundary needed for heterogeneous fused features in imbalanced classes.

Positive ones have been achieved, but there are some limitations still in place that would allow the clinical implementation. This model has been trained and tested on a benchmark that attempts to mimic realistic recording conditions, but may not be representative of the range of device properties and noise levels in the environment and of patient demographics of those in uncontrolled healthcare settings^4,6,9. More importantly, the residual class imbalance can bring bias in favor of majority classes and decrease sensitivity to underrepresented COPD phenotypes, which is why more sophisticated imbalance-aware training methods are being sought. Future research should thus aim at multi-institutional validation with cohorts of clinically collected data, federated learning to better generalization in a privacy limited setting, and explainable AI extensions (e.g. SHAP-based attribution of fused features) to improve interpretability and clinical trust^1,12,14. Altogether, the suggested multimodal concept provides a direction towards scalable and non-invasive respiratory diagnostics and opens a feasible opportunity of enhancing equitable access to respiratory disease screening in low-resource areas.

Comparison of the proposed method with state-of-the-art methods

As a benchmark to the efficiency of our multi-feature fusion model, with the involvement of attention mechanisms, we will compare it with the recently developed state-of-the-art (SOTA) deep learning models, especially those that can be applied to the respiratory sound classification, in particular, asthma, COPD, and most of the related disorders. The majority of previous research uses the ICBHI 2017 dataset (a common benchmark) when the present study evaluates the standardized AIRS Kaggle dataset which recreates the variability of recording presented in reality¹⁵.

The latest developments are hybrid CNN-RNN predictors to learn spatio-temporal characteristics²⁵ of asthma exacerbation by providing greater feature representation in Mel spectrograms²⁶ and enhanced feature capture in dual-channel CNN-LSTM models with respect to single-channel versions, as well as explainable multi-modal models based on Grad-CAM and SHAP to ensure clinical interpretability of predictors across asthma exacerbation, COPD, and additional categories²⁷. Table 6 below analyses the main SOTA approaches of 2024-2025.

Table 6.

Comparison with state-of-the-art methods for respiratory sound classification (2024–2025).

Method	Year	Dataset	Accuracy (%)	Key Features
CNN–RNN Fusion25	2025	ICBHI/Coswara	N/R	Spatio-temporal modeling
Dual-Channel CNN–LSTM26	2024	Respiratory sounds	N/R	Mel-spectrograms, parallel feature streams
Explainable Multi-Modal DL27	2025	Asthma Det. V2	N/R	Grad-CAM, SHAP, multi-modal fusion
CycleGuardian (Deep Clustering)28	2025	ICBHI 2017	63.26/82.06	Lightweight model (38 MB), clustering-based learning
TriSpectraKAN (KAN)29	2025	Lung sound dataset	N/R	Tri-spectral feature extraction
MHSONN + Triplet TFR19	2024	Lung sounds	99.81%	Multi-head self-operational NN, COPD detection
VGAResNet20	2023	Lung sounds	95.13%	Visibility graph adjacency + ResNet, COPD
AsthmaSCELNet21	2023	Lung sounds	98.54%	Supervised contrastive embedding, Asthma
AsTFSONN22	2023	ICBHI dataset	98.50%	Time-frequency self-operational NN, Asthma
Pulmo-TS2ONN23	2024	Respiratory sounds	98.88%	Triple-scale self-operational NN, 7-class, edge deployment
Proposed Model	–	AIRS Kaggle	90.3	Multi-feature fusion, attention mechanism, data augmentation

This model is 90.3 accurate and has a macro F1-score of 0.901 on the AIRS, indicating balanced score across healthy, asthma and COPD classes. This competes well with such approaches as the CNN-RNN fusion model (high class-specific performance on ICBHI)²⁵ and the dual-channel CNN-LSTM (strong overall classification) model²⁶, as well as it enjoys multi-feature fusion and attention to ensure robustness in noisy/simulated settings^13,16,17, which are important features of the CNN-LSTM version. A lightweight design focusing on telemedicine scalability is offered by explainable multi-modal framework with strong interpretability that has multi-modal inputs, but which we do not fulfill.

The proposed model achieves competitive performance (90.3% accuracy, macro-F1 = 0.901) on the standardized AIRS benchmark against recent CNN-RNN²⁵, CNN-LSTM²⁶, and explainable multi-modal²⁷ approaches. While self-operational methods^19–23 achieve higher class-specific accuracy on their respective lung sound datasets (ICBHI, proprietary), they operate under controlled recording conditions and often binary or smaller-class classification settings. The proposed model’s strength lies in (1) balanced macro-averaged performance across three classes, (2) evaluation on a publicly available, competition-standard benchmark, and (3) systematic ablation demonstrating each component’s contribution — making it directly reproducible and comparable for future research.

Limitations of the study

Despite promising results, several limitations merit consideration:

1. Dataset Scope: The model is trained and evaluated on the AIRS Kaggle benchmark, which simulates but may not fully represent real-world acoustic variability across all device types, noise profiles, and patient demographics^4,6,9.

2. Class Imbalance: Residual class imbalance (particularly fewer COPD samples) may limit sensitivity to underrepresented COPD phenotypes despite inverse-frequency weighting^2,3,8.

3. Data Augmentation Scope: While disabling data augmentation reduces performance on our test set, this demonstrates within-dataset benefit only. Proving robustness to channel mismatch or real-world acoustic variability would require evaluation across independent datasets with and without those specific perturbations.

4. Explainability: While attention mechanisms and CatBoost feature importances offer partial interpretability, advanced XAI tools such as SHAP or LIME have not been fully integrated for per-prediction clinical explanation^1,12,14.

5. External Validity: No multi-institutional validation has been performed; external validity is thus constrained to the AIRS competition structure¹⁵. This also constrains direct comparison with SONN methods^19–23 that report substantially higher accuracy on ICBHI or proprietary lung sound datasets.

Conclusion and future work

This paper presents a reproducible, multimodal AI framework for non-invasive respiratory disease screening integrating handcrafted spectral-temporal audio features, precomputed deep cough/vowel embeddings, and structured clinical metadata via feature-level fusion. A class-weighted CatBoost classifier models heterogeneous feature interactions and addresses class imbalance. Evaluated on the standardized AIRS Kaggle benchmark, the framework achieves 90.3% test accuracy and macro-F1 of 0.901, with class-wise F1-scores of 0.945 (Healthy), 0.915 (Asthma), and 0.842 (COPD). MCC (0.856) and Cohen’s Kappa (0.849) further confirm robust classification under imbalanced conditions.

Systematic ablation across all three feature groups and architectural components confirms that multimodal fusion, attention-based segment emphasis, and augmentation-based robustness each contribute measurably to diagnostic performance. Benchmarking against state-of-the-art methods — including high-accuracy self-operational neural network approaches^19–23 — positions the proposed system as a reproducible, publicly benchmarked alternative with strong balanced multi-class performance.

Future directions include: (i) multi-institutional validation on clinically collected cohorts; (ii) federated learning for privacy-preserving generalization; (iii) SHAP/LIME integration for clinical interpretability; (iv) edge/mobile deployment for real-time telemedicine; and (v) extension to multilabel diagnosis of comorbid respiratory conditions.

Author contributions

Javed Rashid: Project administration; Validation; Conceptualization; Resources; Visualization; Writing-original draft; supervision. Turke Althobaiti: Methodology; Validation; Funding; Review & editing; Formal Analysis; Visualization. Asad Ali: Validation; Conceptualization; Resources; Review & editing; Formal Analysis; Visualization, Writing-original draft. Muhammad Shoaib Saleem: Methodology; Supervision; Validation; Visualization; Review & editing; Formal Analysis; Visualization; Hezam Gawbah: Validation; Funding, Resources; Review & editing; Formal Analysis; Visualization. Sultan Hussain: Methodology; Formal Analysis, Conceptualization; Review & editing. All authors reviewed the results and approved the final version of the manuscript.

Data availability

The dataset used in this study is publicly available through the Kaggle competition ‘AIRS – AI in Respiratory Sounds’ at contributed equally to https://www.kaggle.com/competitions/artificial-intelligence-in-respiratory-sounds. All analyses were conducted using the official dataset files and protocol provided by the competition platform. Additional processed features and code can be made available from the corresponding author upon reasonable request.

Declarations

Competing interests

The authors declare no competing interests.