Uncertainty quantification-based DMEFNet for reliable modelling of heart sound signals

Abstract

Phonocardiogram (PCG) analysis is an inexpensive and non-invasive technique for the automatic diagnosis of heart valve diseases. In the clinical domain, PCG recordings are typically corrupted by noise, inter-subject variability, and overlapping signal characteristics, necessitating uncertainty-based decision support. To solve this problem, this paper presents an uncertainty-aware deep multimodal early fusion network (DMEFNet) that jointly integrates one-dimensional(1D) temporal signals and two-dimensional (2D) time–frequency image representations for multiclass PCG classification. Four main uncertainty quantification (UQ) methods, namely Monte Carlo (MC) dropout, Bayesian Neural Networks (BNNs), Deep Ensembles (DE), and Dirichlet-based Evidential Deep Learning (EDL), are used for predictive uncertainty estimation. Extensive experimental evaluations on the public HVD dataset demonstrated that uncertainty estimates are well-calibrated, scoring low predictive uncertainty when samples are correctly classified and higher uncertainty for ambiguous or noise samples. To the best of our knowledge, this work is among the first few attempts to analyze UQ in PCG-based heart valve disease classification. The introduced framework increases clinical trust, reliability and enables risk-aware decisions, thereby promoting the development of PCG-based diagnostic systems for real-world clinical applications.

Introduction

Cardiac auscultation–based diagnostic tools are in growing demand to assist clinical decision-making in emergency situations¹.A phonocardiogram (PCG) is a non-invasive, low-cost technique used to graphically record heart sounds. Automated analysis of PCG signals has gained more attention for early, accurate detection of cardiovascular diseases (CVD), as it contains relevant information about structural abnormalities and valvular disorders of the heart. Recent advancements in deep learning (DL) have made it possible to automatically learning important representations from PCG signals². Convolutional, recurrent, and hybrid network architectures have demonstrated strong capabilities for modelling complex temporal dynamics and subtle acoustic patterns associated with heart problems. This reduces the need for manually crafted features. However, most learning-based diagnostic systems still rely on deterministic inference, yielding a single predicted outcome for each recording. Given the variability in physiology, signal noise, and borderline pathological patterns, these deterministic predictions may not accurately represent the reliability of the diagnostic decision³.

Although existing DL models for PCG-based heart disease classification have achieved impressive performances, most of them make deterministic predictions without providing the quantification of their uncertainty⁴. They pay little attention to two main sources of uncertainty, namely limited data and model parameters (epistemic uncertainty) and inherent noise and variability of PCG recordings (aleatoric uncertainty)⁵. Consequently, when these models are applied to ambiguous or clinically unreliable cases, they may produce overconfident predictions that lead to incorrect diagnostic decisions. Subtle acoustic cues such as murmurs, splits and transient anomalies in heart sound are clinically important in diagnosis⁶. However, blindly trusting the results from machine learning and DL models can lead to sensitive signal characteristics being lost or misinterpreted. The lack of understanding of how to calculate uncertainty from model predictions by these DL models is inevitable. As they are often represented as ‘black boxes’.

Inspired by these drawbacks, this work incorporates uncertainty quantification (UQ) into deep learning–based PCG classification. Uncertainty is a measure of how confident or unsure the model is about its own predictions, especially when the input signal is noisy, ambiguous, or different from what it already shown⁷. To address the issues of overconfident predictions and the reliability of deterministic models in clinical decision-making, we present an uncertainty-aware deep multimodal early fusion network (DMEFNet) to estimate predictive uncertainty from complementary PCG representations. Motivated by various UQ techniques and deep learning models, we propose a hybrid UQ model based on DMEFNet. Building on this, we will first employ different UQ techniques and then estimate the predictive uncertainty of DMEFNet. As we are aware, this work is among the first to systematically consider uncertainty quantification for heart valve disease classification using PCG, thereby making our model more robust and transparent and readily adaptable in clinical practice.

While deep learning models for PCG classification achieve high accuracy, they often neglect uncertainty estimation and reliable prediction confidence. To mitigate the issues, we introduce an uncertainty-aware DMEFNet to improve the robustness of PCG automatic diagnosis. The main contributions of this work are summarized as follows:

• Development of a comprehensive UQ-aware framework to ensure reliable heart valve disease classification using PCG signal.

• Four well-known UQ techniques are systematically applied: Monte Carlo (MC) dropout, Bayesian neural network (BNN), Deep Ensemble (DE), and Evidential Deep Learning (EDL).

• Implementation of proposed study on a previously validated deep multimodal early fusion network (DMEFNet) to evaluate predictive performance and model confidence.

• Extensive evaluation of the proposed model using calibration and sharpness metrics to provide an in-depth understanding of uncertainty behavior.

• Robust analysis under various noise conditions is performed to showcase real-world clinical scenarios and to evaluate the reliability of model predictions and uncertainty estimates.

Related work

In this section, we present a brief review of recent studies on the application of UQ to biomedical data analysis and on deep learning techniques for heart valve disease classification.

Uncertainty quantification in biomedical data

Uncertainty quantification (UQ) is a key component of reliable artificial intelligence, especially in biomedical applications, where incorrect predictions can lead to serious issues. In deep learning models, uncertainty is usually divided into aleatoric uncertainty, which comes from natural data noise, and epistemic uncertainty, which is caused by having limited training data or uncertainty in the model⁸. UQ techniques are used to mitigate high-risk scenarios in medical diagnosis by providing reliable and robust confidence in model predictions. It has been more explored in biomedical image applications like chest X-rays⁹, MRI brain images¹⁰, CT scan images⁷ and Skin lesion images¹¹ etc. In all these existing studies, several UQ techniques like Monte Carlo dropout, Bayesian models, deep ensemble methods and test time augmentation are employed for determining the uncertainty of the model architectures.

Apart from imaging, UQ has also implemented biomedical signal data, such as electrocardiograms (ECG) and electroencephalograms (EEG). In¹², the authors used the Monte Carlo method to evaluate the performance of ECG data. An uncertainty-aware deep learning model was designed for ECG classification in¹³. The study employs max-pooling residuals to capture spatiotemporal features and subjective logic to quantify the model’s uncertainty. In a recent work,¹⁴ introduced a Trusted Reliable Uncertainty Estimation (TRUEE) framework for EEG. Multiple features are extracted from the EEG by decomposing it and deep learning networks are used for learning, achieving an accuracy of 86.5%. The TRUEE framework provided a reliable system through subjective logic-based uncertainty quantification.The effectiveness of uncertainty-aware and adversarial domain adaptation strategies for EEG-based depression recognition has been investigated in recent studies such as UA-DAAN¹⁵ and WDANet¹⁶, which aim to enhance robustness to different data distributions. These methods improve generalization and reliability in cross-domain biomedical signal analysis by employing uncertainty estimation and distribution alignment using Wasserstein distance-based adversarial learning.

Deep learning based heart valve disease classification

Several computational methods have been suggested to improve the reliability and accuracy of PCG-based cardiac assessment. These range from traditional feature-based approaches to modern deep learning systems. Most of the studies utilised the advantages of Mel frequency cepstral coefficients (MFCC)¹⁷, wavelet transform^18,19, Fourier transform²⁰, and empirical mode decomposition (EMD)²¹ etc, for extracting features with intricate spatial patterns and temporal dependencies, thus detecting various pathological heart diseases. With the emergence of convolutional neural network (CNN)- based deep learning architectures such as VGG16, LeNet-5, AlexNet, and GoogleNet, which automatically learn features from data and classify heart valve disease by separating normal signals from abnormal signals^22,23. These techniques paved the way for automated diagnosis of heart diseases through efficient, real-time PCG signal processing. Even though these studies highlight significant improvements, they fail to demonstrate the confidence and reliability of the results. To analyse the temporal dependencies of PCG, some authors have introduced recurrent neural network-based studies that specifically implement long short-term memory (LSTM)- related analyses. By incorporating digital signal processing with an LSTM, the authors of²⁴ established an automated heart sound classification system with real-time processing. Apart from all this, several studies have proposed hybrid network architectures for PCG signal analysis, combining CNNs with recurrent networks such as CNN-GRU²⁵, CNN-LSTM²⁶, and CNN-BiLSTM²⁷ etc. By utilizing hybrid network architectures, the model leverages the advantages of both networks to efficiently extract the prominent characteristics of PCG, thereby identifying heart valve abnormalities. Despite these advances, existing deep learning-based PCG studies primarily rely on deterministic modeling assumptions and provide point predictions without accounting for predictive uncertainty. These models can generate very confident outputs, even for noisy or unclear signals. This limits their reliability in real-world clinical situations.

It is clear from the literature that deep learning has significantly improved PCG classification performance, while the lack of uncertainty-aware modeling compromises the clinical reliability and trustworthiness of existing studies. On the contrary, although UQ has proved its great worth in a broad range of biomedical applications, its systematic role in PCG-based cardiac diagnosis is underexplored. Motivated by this gap, the proposed work focuses on integrating uncertainty quantification with a deep multimodal fusion network to enable transparent and robust heart valve disease classification. Thus, improves predictability, confidence, interpretability, and applicability of the proposed novel architecture for clinical decision support.

Proposed methodology

The proposed UQ framework along with the baseline model is explained in the following section. We provide a general explanation of the baseline model, the theoretical background of UQ with the proposed DMEFNet-based UQ model, and the synthesis of the PCG signal by adding noise. Thereafter, the uncertainty evaluation metrics used in the proposed method are explained.

Baseline model (DMEFNet)

In this work, the uncertainty quantification analysis is carried out on the same deep multimodal early fusion network (DMEFNet) developed in our previous study²⁸. DMEFNet integrates a 1D temporal PCG signal and a corresponding 2D time–frequency representation for heart valve disease classification. The 1D branch utilizes a hybrid CNN–BiGRU architecture to process the raw PCG signal. The convolutional layers play a primary role in extracting local temporal features, including waveform morphology and short-duration acoustic events, and the bidirectional gated recurrent unit (BiGRU) captures long-range temporal dependencies by processing the signal in both forward and backward directions. It facilitates an efficient modeling of the non-stationary and sequential nature of heart sounds. In the 2D branch, through Synchrosqueezed Transform (SST)²⁹ PCG is converted to time–frequency representation. Based on instantaneous frequency estimates, the wavelet coefficients are reallocated, thereby enhancing time–frequency resolution and identifying more subtle spectral characteristics, increasing class separability. The proposed SST-based images are then processed by a pretrained DenseNet-201 architecture that leverages efficient feature extraction through dense connectivity and improved gradient flow. To better discriminate features, a lightweight self-attention mechanism is introduced into both branches, enabling the model to focus on more informative regions of the signal and thereby improving representation learning.

Early fusion mechanism is then applied to combine features extracted from both the 1D and 2D branches. More precisely, the feature vectors of both modalities are concatenated to create a combined representation:

1

Where and be the feature vectors extracted from the 1D PCG and 2D PCG-SST branch, respectively²⁸. This fused feature vector incorporates both temporal and spectral features of PCG signals and is then classified by fully connected layers. This architecture allows the model to effectively learn multimodal features by exploiting complementary information across both modalities, thereby improving the overall performance in terms of capturing complex patterns correlated with different cardiac conditions. Figure 1 illustrates the overview of the baseline model.

Figure 1.

Overview of baseline model (DMEFNet)²⁸.

Uncertainty quantification methods

Uncertainty Quantification (UQ) is the process of computing how confident a model is in its predictions, rather than relying solely on deterministic outputs. In medical-based decision support systems, UQ is important for detecting uncertain or unreliable cases to support risk-aware decision-making and enhance clinical trust in automatic models³⁰.

UQ is divided into two types: aleatoric and epistemic uncertainty. Aleatoric uncertainty refers to the inherent uncertainty in data arising from noise, variation, or ambiguity in input signals. This involves changes arising from patient physiology, recording conditions, sensor noise and background interference in PCG analysis. Epistemic uncertainty accounts for uncertainty in the model parameters and structure, which is generally induced by limited training data or limited knowledge of the model. This is distinct from aleatoric uncertainty, which can be reduced through more or varied training samples and better model architecture. The four well-known UQ methods which are proposed in this study are discussed below:

Monte Carlo (MC) dropout

Monte Carlo (MC) dropout is an easy-to-use and widely used method for estimating epistemic uncertainty in deep learning models. It is an approximation to Bayesian inference over the parameters of the model and achieves this by stochastically dropping network units during forward passes, without having to specify an explicit probabilistic model³⁰. During inference, dropout is maintained and the model is run on one input multiple times (using a stochastic forward pass). Each pass yields a distinct prediction due to random dropout masks, so an output distribution exists rather than a single deterministic prediction. This distribution accounts for the model’s uncertainty, specifically uncertainty arising from limited data or model information.

For a given input x, let indicates the prediction of stochastic forward pass. The predictive mean estimated for T MC samples is given by;

2

which is the final prediction. Whereas predictive variance, a measure of epistemic uncertainty is calculates as;

3

For classification tasks, we have class probability vectors for each MC sample. The average predicted probability is estimated as;

4

and uncertainty can be measured with variance or entropy of the mean predictive distribution. MC Dropout provides an efficient and scalable way of estimating uncertainty, which does not necessitate any architectural changes but introduces only a minor computational overhead.

Bayesian neural network (BNN)

Bayesian Neural Networks (BNNs) treat model parameters as random variables but not as fixed ones and hence ensure uncertainty estimates related to the predictions. Rather than learning a unique optimal weight, BNNs learn a probability distribution over network parameters, enabling the model’s uncertainty to be explicitly quantified⁵. This probability modelling allows BNNs to express epistemic uncertainty arising from data scarcity or partial ignorance of model parameters, which is crucial in many safety-critical applications, including medical diagnosis.

Consider a data , the posterior distribution over the network parameters (θ) is computed through Bayesian inference³¹:

5

where (p(θ )) denotes the prior distribution on parameters reflecting prior beliefs about their values, represents the likelihood of the data given each parameter set, and (p(D)) corresponds to marginal likelihood (evidence). Given a new input , the predictions are formed by marginalizing over the posterior of parameters³¹:

6

This predictive distribution naturally encapsulates uncertainty in the model output. By explicitly representing uncertainty in network parameters, BNNs deliver trustworthy and interpretable predictions, unlike deterministic neural networks, and are thus well-suited for uncertainty-aware medical decision-support systems.

Deep ensemble (DE)

Deep Ensembles are an extremely effective and scalable way to estimate uncertainty in deep learning. The main concept is to train several neural networks independently with distinct random initialization, data shuffling, or bootstrap samples³². This approach captures both aleatoric and epistemic uncertainty, arising from data noise, or due to model limitations or insufficient data. DE will collect predictions from independent training models and average them to estimate uncertainty. Deep Ensembles provide strong uncertainty estimates without any Bayesian inference and change in architecture. Owing to their simplicity, robustness and empirical strength, they have become a popular choice for uncertainty-aware deep learning applications³².

Evidential deep learning (EDL)

In Evidential deep learning (EDL), the model is trained to minimize a loss function that promotes accurate predictions, as well as regularizing uncertainty³³. This is obtained by adopting a Dirichlet-based data-fitting term, combined with Kullback–Leibler (KL) divergence regularization.

For an inputx, let dirichlet parameters predicted by the network be . Let be the one-hot encoded ground-truth label.The Dirichlet expected mean squared error loss is the data-fitting term that minimizes the expected (over the training set) squared error with respect to true labels is given by³³:

7

where

8

The first term acts as a penalty for predicting incorrectly and the second term reflects uncertainty on the Dirichlet distribution. To avoid the model from generating too much or unjustified evidence, a KL divergence term is employed for regularizing the learned Dirichlet distribution to lack information compared with its prior³³.

9

where Dir(1) indicates uniform Dirichlet prior as maximum uncertainty. The overall EDL loss is given by;

10

where λ is a regularization parameter that controls the trades off between the prediction accuracy and uncertainty calibration.

As shown in Fig. 2, UQ is incorporated into the DMEFNet to assess the reliability of predictions from a deep learning–based multiclass PCG classification model. The advantage of the proposed method is that, unlike classical discriminative classification methods, it evaluates predictive performance and associated uncertainty simultaneously, enabling more accurate and clinically interpretable decision-making. As the first phase, two separate modalities of the PCG signal are classified using DMEFNet. To assess the confidence and trustworthiness of the DMEFNet model, various UQ methods are employed. Four well-known UQ methods, such as MC dropout, BNN, DE, and EDL, are implemented and tested on multiclass heart-valve PCG data. To comprehensively evaluate prediction confidence and uncertainty quality, a range of classification, calibration, and sharpness metrics is used.

Figure 2.

Illustration of the proposed UQ estimation framework for DMEFNet.

Adding noise to proposed DMEFNet with UQ estimation

To test the authenticity of the DMEFNet with the UQ estimation model, we are synthesizing the input PCG signal and PCG-SST image with some noise that is generally present. As illustrated in Fig. 3, two separate noise modules are added to the 1D PCG signal and the 2D PCG-SST image. Two noises, like pink noise and powerline hum, are added to the 1D PCG modality. Pink noise³⁴ mimics low-frequency physiological and environmental disturbances, whereas powerline hum³⁵ corresponds to power-line or electrical interference that arises during the recording of biomedical signals. For the 2D PCG modality, multiplicative speckle noise³⁶ and motion blur are added to the SST-based time-frequency images to simulate amplitude-dependent distortions and temporal smearing effects, respectively. These noise components are chosen to represent realistic degradations that impair both the time and spectral drawings of PCG signals.

Figure 3.

Schema employed for noise addition in the proposed UQ-based DMEFNet.

In this work, the SNR is adjusted in the range from 15 dB to 30 dB, representing low-to-moderate noise levels that degrade diagnostically relevant heart sound features yet realistically challenge our model. This range represents typical auscultation scenarios and is chosen to avoid very low SNR cases, which correspond to highly corrupted signals with very limited clinical content. The noisy input signals are fed into the same DMEFNet pipeline, and uncertainty estimation is performed using MC Dropout, BNN, DE, and Dirichlet EDL. Then, UQ estimation is done using classification, calibration and sharpness metrics. Employing the proposed method across the given noise conditions is essential for understanding the robustness and reliability of the uncertainty-aware framework proposed here, thus justifying its applicability to real-world clinical deployment.

Bayesian optimization and cross validation

Bayesian optimization and cross-validation are used together to efficiently tune the hyperparameters and evaluate the models. Cross-validation removes bias from a single train–test split, guaranteeing reliable performance estimates, and Bayesian optimization efficiently tunes the hyperparameter space of extremely time-consuming DL models³⁰. In the proposed method, Bayesian optimization suggests hyperparameter configurations sequentially based on a surrogate probabilistic model and an acquisition function. For each configuration, the average validation performance is evaluated using K-fold cross-validation, and these values are reported as the objective. This procedure is applied until the optimal hyperparameters are found. The combination of cross-validation and Bayesian optimization also enhances model generalization and repeatability, which is advantageous for biomedical signal analysis when sample sizes are limited.

Experimental validation

This section presents the PCG dataset used in the proposed work, the experimental setup of the UQ framework with DMEFNet, and the evaluation metrics used for performance and uncertainty estimation. Finally, the experiment and the obtained results are presented and discussed.

Dataset and experiment setup

The dataset used for the proposed work is a standard benchmark, the Yaseen heart valve disease (HVD) dataset³⁷. HVD data are widely used PCG benchmarks for evaluating and comparing heart valve disease classification frameworks. It consists of five classes, namely AS (Aortic Stenosis), MS (Mitral Stenosis), MR (Mitral Regurgitation), MVP (Mitral Valve Prolapse), and N (Normal), with a sampling rate of 8000 Hz. The HVD data contains 1000 PCG recordings, with each class comprising 200 recordings. The data is preprocessed with a fourth-order Butterworth band-pass filter (25 Hz to 900 Hz) to remove artifacts.

All experiments are conducted using Jupyter Notebook with deep learning libraries, such as TensorFlow/Keras and PyTorch etc, on a high performance computing system NVIDIA A100 GPU (80 GB VRAM) and 32 GB of RAM. In each fold, the dataset is split using an 80%-20% stratified approach to ensure class distribution. The training procedure and validation of the DMEFNet model used in this study are presented in our prior work²⁸. In this study, to perform uncertainty analysis, the same validated model is used without modification. Training model is implemented with Adam optimizer and categorical cross-entropy loss, learning rate of 0.001, batch size of 16, dropout rate of 0.5 and L2 regularization strength of . The training is run for a maximum of 30 epochs, and early stopping on the validation loss is used to prevent overfitting. Our hyperparameter choice is controlled by Bayesian Optimization. For uncertainty estimation, stochastic predictions are made at test time by sampling 25 stochastic forward passes for MC dropout and BNN, while DE fuses predictions from multiple independently trained models, and EDL estimates uncertainty directly from the evidence learned. Table 1 summarizes the uncertainty evaluation metrics employed in the proposed work³⁸.

Table 1.

Predictive uncertainty evaluation metrics³⁹³⁸. TC: true certainty (certain and correct), TU: true uncertainty (uncertain and incorrect), FU: false uncertainty (uncertain and correct), FC: false certainty (certain and incorrect). : input sample, : predicted probability of true class , : predicted probability of class k, : predicted probability that sample i belongs to class k, : ground truth label, : confidence bin, : bin accuracy, : bin confidence, N: number of samples, K: number of classes, M: number of bins, : prediction at stochastic forward pass, T: total stochastic passes.

Metric Category	Metric and Description	Mathematical Expression
Uncertainty performance metrics	Uncertainty Accuracy (UAC): Proportion of correct outcomes to the total number of predictions.
	Uncertainty Precision (UPR): Ratio f uncertain and incorrect predictions.
	Uncertainty Sensitivity (USE): Proportion of incorrect predictions that are correctly identified as uncertain.
	Uncertainty Specificity (USP): Proportion of correct predictions that are correctly identified as certain
Calibration Metrics	Negative Log-Likelihood (NLL): Measures how well predicted probabilities explain true labels
	Brier Score: Mean squared error between predicted probabilities and ground truth
	Expected Calibration Error (ECE): Difference between predicted confidence and empirical accuracy
	Maximum Calibration Error (MCE): Worst-case calibration error across all confidence bins
Sharpness Metrics	Predictive Entropy: Measures total uncertainty in predictions
	Mutual Information (MI): Quantifies epistemic uncertainty due to model parameters
	Predictive Variance: Measures dispersion of predictions across stochastic samples

Experimental results

The results obtained from the experiment using the proposed work are discussed in the sections below. The experiments are done on clean PCG data and noisy PCG data (SNR: 15dB to 30dB)

UQ estimation in clean PCG data

From Table 2, it is observed that all UQ methods yielded high uncertainty estimates. High values of UAC, UPR, USE, and USP indicate that the model was informed of what it understood (correct classification with high certainty) and what it did not know (uncertain about missclassification). Thus demonstrating the efficiency of the DMEFNet model when a clean PCG signal is available. For example, MC Dropout achieves 99.99% performance across all uncertainty performance metrics, indicating almost perfect separation between cardiac classes. BNN also achieves 99.50% accuracy, while Deep Ensemble and Dirichlet-based EDL achieve over 98.5% accuracy, showing strong baseline performance across all UQ methods.

Table 2.

Classification performance (%) obtained for different UQ methods.

Metrics	MC dropout	BNN	DE	EDL
UAC	99.99	99.50	98.50	98.50
UPR	99.99	99.53	98.56	98.53
USE	99.99	99.50	98.63	98.50
USP	99.99	99.62	99.50	99.62

Beyond classification metrics, Table 3 summarizes calibration metrics for the UQ methods employed. Calibration metrics provide important information about how much we can trust the model’s predicted probabilities. MC dropout has the lowest NLL of 0.0590 and Brier score of 0.0193, indicating that its predicted confidence values are very close to the true class labels. DE and EDL, on the other hand, have larger NLL values (0.3346 and 0.3321), indicating more diffuse predictive distributions even under clean input. This is emphasized by the Expected Calibration Error (ECE): while MC dropout has an ECE of 0.0407, we obtain significantly higher ECEs of 0.2562 and 0.2412 for DE and EDL, which yield rather conservative but less sharply calibrated confidence estimates. Even though DE and EDL yielded slightly higher values than MC and BNN, they indicate low uncertainty and high confidence in model performance. If the values are near 0, the model is less uncertain and highly confident, whereas if the values are near 1, the model exhibits high uncertainty and low confidence.

Table 3.

Calibration metrics obtained for various UQ methods.

Metrics	MC dropout	BNN	DE	EDL
NLL	0.0590	0.06230	0.3346	0.3321
Brier score	0.0193	0.0241	0.1169	0.1142
ECE	0.0407	0.0507	0.2562	0.2412
MCE	0.5204	0.5672	0.4901	0.5543

Table 4 summarizes sharpness metrics that measure how narrow the predictive distributions become when the input signal is unambiguous. Under clean conditions, MC Dropout and BNN have low predictive entropy (0.1540 and 0.1731) and little predictive variance (0.0023 and 0.0040), implying confident signals and stable predictions. Their mutual information values (0.0050 for MC Dropout and 0.0327 for BNN) are low, indicating low epistemic uncertainty when the models have sufficient information to make confident decisions. Both Deep Ensemble and EDL values were higher compared to the other two methods by an order of magnitude for entropy (0.3256 and 0.3246, respectively) and mutual information (0.3120 and 0.3241, respectively), indicating that they intentionally put more conservative estimates on the uncertainty even in the case where no noise was present. Although their values are higher than MC and BNN, they exhibit less uncertainty because they are closer to 0 than to 1.

Table 4.

Sharpness metrics obtained for various UQ methods.

Metrics	MC dropout	BNN	DE	EDL
Entropy	0.1540	0.1731	0.3256	0.3246
MI	0.0050	0.0327	0.3120	0.3241
PV	0.0023	0.0040	0.0263	0.0254

From a clinical point of view, these results validate that the proposed framework can achieve good performance while efficiently separating certain and uncertain samples. This will enhance the identification of uncertain samples, leading to missclassification, which is critical for reliable decision-making in heart disease analysis.

Reliability of UQ estimation against noise

Table 5 contains the uncertainty performance metrics of various UQ methods as the noise level increases (15-30 dB SNR). As the signal degrades to 15 dB, the uncertainty performance decreases, and the missclassification rate increases. This decreases as the signal-to-noise ratio reaches 30 dB. MC Dropout and BNN exhibit strong robustness at medium noise levels, while DE and Dirichlet-based EDL show less vulnerability. Most methods maintain high USE even at low SNRs, which is crucial in clinical practice, where they should correctly detect pathological heart sounds amid noise. High USP at a high SNR indicates fewer false alarms in clean conditions. From a clinical perspective, as noise increases, uncertainty-aware models decrease confidence but may make mistaken predictions, thereby reducing the risk of false clinical decisions. In addition, the gradual performance degradation, rather than catastrophic failure, demonstrates that the proposed model is stable and reliable under practical conditions for heart valve disease diagnosis.

Table 5.

Summary of performance obtained for different UQ methods under noisy conditions (15–30 dB SNR).

UQ Method	SNR (dB)	UAC	UPR	USE	USP
MC Dropout	15	0.9400	0.9538	0.9850	0.9400
	20	0.9750	0.9766	0.9938	0.9750
	25	0.9850	0.9856	0.9963	0.9850
	30	0.9980	0.9975	0.9985	0.9980
BNN	15	0.9100	0.9775	0.9300	0.9100
	20	0.9705	0.9925	0.9741	0.9700
	25	0.9852	0.9930	0.9865	0.9850
	30	0.9930	0.9950	0.9950	0.9900
DE	15	0.9400	0.9492	0.9400	0.9850
	20	0.9750	0.9767	0.9750	0.9938
	25	0.9800	0.9810	0.9800	0.9950
	30	0.9820	0.9836	0.9820	0.9843
EDL	15	0.9100	0.9529	0.9572	0.9824
	20	0.9530	0.9763	0.9758	0.9828
	25	0.9790	0.9817	0.9850	0.9850
	30	0.9830	0.9820	0.9840	0.9832

From Table 6 for a noise range of 15-30 dB, the metrics NLL and Brier Score both increase as SNR decreases, suggesting that the predicted probabilities become increasingly inconsistent with the true labels in the presence of noise. ECE quantifies the average deviation in a classifier’s confidence from its empirical accuracy, while MCE measures worst-case calibration. Lower values for these metrics indicate better-calibrated uncertainty estimates. BNN and MC Dropout also have lower ECE and NLL than DE and EDL at a moderate noise level, indicating better probabilistic confidence under moderate noise. Higher calibration errors at lower SNRs, however, represent proper model behavior in light of the fact that predictions become more uncertain with increasing input ambiguity. A well-calibrated model prevents overconfident predictions under low signal quality and alerts clinicians to cases that require further scrutiny rather than blindly trusting automatically provided outputs. Each calibration metric captures a different but complementary aspect of uncertainty. NLL reflects the quality of probabilistic prediction, whereas ECE measures the degree of alignment between reliability and accuracy. A model that is confident in its predictions can achieve low NLL but exhibit poor calibration. Hence, both kinds of metrics are required for full-scale assessment.

Table 6.

Calibration metrics obtained for different UQ methods under noisy conditions (15–30 dB SNR).

UQ Method	SNR (dB)	NLL	Brier Score	ECE	MCE
MC Dropout	15	0.4212	0.1135	0.0746	0.4378
	20	0.0954	0.0397	0.0625	0.5621
	25	0.0725	0.0259	0.0536	0.5969
	30	0.0665	0.0272	0.0495	0.4382
BNN	15	0.2517	0.1332	0.0509	0.7431
	20	0.1103	0.0486	0.0483	0.6167
	25	0.0851	0.0386	0.0400	0.5912
	30	0.0748	0.0327	0.0362	0.5241
DE	15	0.4544	0.1869	0.2652	0.4792
	20	0.3695	0.1352	0.2609	0.4720
	25	0.3488	0.1243	0.2603	0.4304
	30	0.3405	0.1201	0.2581	0.4324
EDL	15	1.1698	0.1519	0.2851	0.6324
	20	0.5498	0.1502	0.2781	0.6214
	25	0.5260	0.1436	0.2729	0.5873
	30	0.4158	0.1290	0.2482	0.5621

Table 7 shows that predictive entropy increases as SNR decreases, and therefore, uncertainty in the predicted models increases with noise corruption in PCG signals.MI, a measure that primarily reflects epistemic uncertainty, captures model uncertainty due to a lack of confidence in learned parameters, as observed in Bayesian and ensemble approaches. The predictive variance also captures variation across stochastic forward passes or across ensemble members. DE and Dirichlet-based EDL have higher entropy/MIs values, especially at lower SNRs, indicating greater sensitivity to noise-induced uncertainty. Both MC Dropout and BNN exhibit relatively modest increases in uncertainty, indicating that they provide rather consistent uncertainty estimates under mild noise. From a clinical point of view, this feature is valuable: when the signal is of poor quality, measurement becomes less certain, which intuitively reflects on test reliability. “Such uncertainty-aware responses can avoid incorrect automatic diagnoses and guided risk-aware clinical workflows by triggering manual inspection or additional examination.

Table 7.

Sharpness metrics obtained for different UQ methods under noisy conditions (15–30 dB SNR).

UQ Method	SNR (dB)	Entropy	MI	PV
MC Dropout	15	0.3345	0.0578	0.0612
	20	0.2231	0.0380	0.0583
	25	0.1946	0.0315	0.0325
	30	0.1770	0.0182	0.0110
BNN	15	0.3292	0.0429	0.0057
	20	0.2178	0.0395	0.0054
	25	0.1722	0.0356	0.0045
	30	0.1636	0.0263	0.0037
DE	15	0.6995	0.3119	0.0267
	20	0.5295	0.3116	0.0265
	25	0.4435	0.3113	0.0264
	30	0.3470	0.3064	0.0263
EDL	15	0.6824	0.3845	0.0299
	20	0.5421	0.3741	0.0284
	25	0.4952	0.3657	0.0267
	30	0.3512	0.3451	0.0241

The practical applicability of various UQ methods is assessed not only based on predictive performance and uncertainty evaluation, but also in terms of their computational cost; as shown in Table 8, the inference time per sample of the baseline DMEFNet model was 450 ms with a model size of 87.90 MB, whereas EDL maintains similar efficiency to enable real-time application deployment. While MC Dropout and BNN require a series of stochastic forward passes, which increases inference time, and DE requires multiple model instances, adding to both training time and memory cost. We face a clear trade-off between computational efficiency and uncertainty estimation capability, while sampling- or ensemble-based approaches can provide richer uncertainty estimates at additional cost.

Table 8.

Overall computational cost of UQ methods.

Method	Training Time (min)	Inference Time (ms/sample)	Model Size (MB)
Baseline (DMEFNet)	∼6–10	450	87.90
MC Dropout (Clean)	∼6–10	∼9000	89.32
MC Dropout (Noise)	∼6–10	∼9500	89.32
BNN (Clean)	∼10–20	∼9000–10000	∼175.80
BNN (Noise)	∼10–20	∼9500–10500	∼175.80
DE (Clean)	∼20–35	∼2250	439.50
DE (Noise)	∼20–35	∼2500	439.50
EDL (Clean)	∼7–10	450	89.89
EDL (Noise)	∼7–10	∼ 500	89.89

Discussions

In this section, the advantages and relevance of the proposed model are explained. Also, the importance of the proposed model in heart disease diagnosis is discussed.

Proposed model evaluation

The following subsections discuss the evaluation of the proposed model in clean and noisy PCG data.

Clean PCG data

Figure 4 shows the uncertainty confusion matrices for clean PCG data for MC Dropout, BNN, DE, and EDL, indicating accurate multiclass classification of the AS, MR, MS, MVP, and Normal classes. MC Dropout exhibits almost perfect separation, with a single missclassification in the MR class, indicating sharp, confident predictions under clean conditions, which emphasizes its low predictive entropy and mutual information. BNN shows comparable performance and slight confusion between acoustically related murmur classes like MR and MVP, which can be attributed to its epistemic uncertainty. DE and EDL also retain high classification performance, with only a few misclassifications observed (mainly within the MR class, a clinically very challenging group due to overlapping murmur characteristics). DE and EDL tend to exhibit more variability in their uncertainty, but this is due to model disagreement and evidence uncertainty rather than diminished performance. Clinically, this suggests all UQ methods with clean PCG data provided robust predictions for the model and they have complementary strengths in terms of their uncertainty profile for decision support tasks such as confidently diagnosing cases and identifying those requiring further clinical evaluation.

Figure 4.

Uncertainty confusion matrices obtained for clean PCG data using different UQ methods.

Noise PCG data

Confusion matrices at an SNR of 30 dB for the proposed study with different UQ methods are illustrated in Fig. 5. In contrast to the clean condition, noise results in a limited number of misclassifications between inter-pathological classes with similar acoustic traits (AS–MR/MR–MVP). MC Dropout and BNN are characterized by strong diagonal dominance, showing robust predictions even under low-quality degraded signals, with only a small number of isolated errors in the MR class. DE and EDL models have a slightly larger missclassification with uncertain prediction. Both the Normal and MS classes under-perform across all methods, indicating that the learned representations remain robust across these two categories. From a clinical point of view, the limited off-diagonal errors combined with high true certainty indicate that the proposed uncertainty-aware fusion framework preserves even under realistic noise levels with good discrimination power, and that the missclassification occur at class boundaries, which are also points to be resolved in practice.

Figure 5.

Uncertainty confusion matrices for noisy PCG data at 30 dB using different UQ methods.

Figures 6 and 7 shows the calibration and sharpness metrics vs SNR plots obtained for various UQ methods respectively. The results of the calibration metrics (ECE, NLL, and Brier Score) demonstrate a clear improvement with increasing SNR: the predicted probabilities become better aligned with the correct labels. However, sharpness metrics (Entropy, MI and PV), showcases positive correlation with less noise which indicates more confident and focused predictive outputs as SNR rises. In general, the predictions from MC Dropout and BNN are sharper and well-calibrated, making them more reliable compared to DE and EDL that remain with high uncertainty, especially in noisy scenarios.

Figure 6.

Plots of calibration metrics vs. SNR (db) obtained for various UQ methods.

Figure 7.

Plots of sharpness metrics vs. SNR (db) obtained for various UQ methods.

State-of-the-art comparison

The proposed multimodal PCG classification framework is compared with state-of-the-art results reported on the HVD database, as shown in Table 9. A few studies in the literature have shown good classification performance using hand-designed signal descriptors (such as local energy, entropy, and TQWT-based features), DL representations, or feature-level fusion approaches with CNN, BiLSTM, or attention-based architectures. However, these frameworks are essentially deterministic classifiers and do not address the model’s prediction reliability or confidence. Most importantly, existing studies lack in providing a quantitative evaluation of model calibration or uncertainty estimation. As these are crucial in clinical auscultation, where PCG recordings may be noisy and subject to inter-patient variability, and overlapping murmur features of varying severity can lead to overconfident and incorrect predictions. This leaves open the question of whether these high-performing models are well-calibrated or perhaps overconfident for different signal conditions.

Table 9.

Comparison of existing PCG studies in terms of uncertainty awareness using HVD dataset.

Study	Features and Method	UQ	UQ Technique
Ghosh et al.40	Local energy, local entropy with MCC	✗	–
Chui et al.41	Deep learning features with WaveNet	✗	–
Baghel et al.42	Deep features with CNN	✗	–
Zeng et al.43	TQWT + EMD + Shannon entropy with Neural Network	✗	–
Alkhodari and Fraiwan44	MODWT features with CNN-BiLSTM	✗	–
Shuvo et al.45	Feature-level fusion with Bi-LSTM	✗	–
Mekahlia et al.6	Scalogram images with CNN	✗	–
Chen et al.46	Deep features with MLP	✗	–
Carter et al.47	Mel-spectrogram coefficients with CNN	✗	–
Ranipa et al.48	Feature-level fusion with MACNN	✗	–
Proposed Work	Multimodal PCG representation (1D + 2D) with early fusion	✓	MC Dropout, BNN, Deep Ensemble, EDL

✓ – presence of UQ; ✗ – absence of UQ. MCC – Multiclass Composite Classifier, TQWT – Tunable Q-factor Wavelet Transform, EMD – Empirical Mode Decomposition, DAG – Directed Acyclic Graph, MACNN – Multimodal Attention-based CNN.

To address this shortcoming, the proposed study conducts a systematic uncertainty-aware analysis by fusing and comparing different UQ techniques for PCG-based heart valve disease classification, providing additional insight into model robustness beyond standard performance measures. The proposed methodology incorporates UQ into a deep multimodal early fusion architecture that jointly processes 1D temporal PCG signals and 2D time–frequency image representations. Despite achieving a higher uncertainty performance metrics of 99.99%, the proposed method explicitly captures predictive uncertainty, which is absent in existing work. To the best of our knowledge, this work is among the first to incorporate UQ into PCG-based heart valve disease classification using the HVD database. Clinically, this is important because it allows the system to distinguish between certain and uncertain predictions, enabling misclassified cases to be presented for clinical review rather than high-risk automated decisions. In summary, the comparison demonstrates that the proposed framework improves model prediction confidence and diagnostic fidelity for clinical trustworthiness, which is not possible with previous deterministic PCG models which lack uncertainty estimation.

UQ significance in proposed study

Although UQ methods have been investigated in other biomedical domains, such as ECG and EEG signals, the contribution of UQ techniques to PCG-based heart sound classification has not yet been extensively studied. This suggests a significant gap in the existing literature, as most models are evaluated solely by accuracy without accounting for prediction confidence. UQ is essential for improving the reliability and clinical use of the developed DMEFNet. The integration of UQ enables the baseline model to explicitly represent the confidence (certainty and uncertainty) associated with each diagnostic decision, whereas conventional deterministic models provide only point predictions. This is particularly useful in the analysis of PCG, where heart sound recordings often suffer from high noise contamination, inter-subject variation, recording artifacts, and overlapping acoustic patterns across different valve diseases. The developed framework can distinguish between confident predictions and ambiguous cases by modeling predictive uncertainty, thereby mitigating the risk of overconfident misdiagnosis.

The proposed UQ framework is employed to capture both data-inherent uncertainty and model-related uncertainty as a supplement to the baseline DMEFNet model. The uncertainty-aware predictions enable the model to modulate its confidence based on input quality: under noisy conditions, greater uncertainty is observed, whereas for clean inputs, less uncertainty arises. Clinically, we consider this behavior highly desirable because it helps the clinician become more risk-aware by properly discarding low-confidence (uncertain and misclassified) predictions in favor of additional clinical examination or a follow-up diagnostic test rather than issuing a false diagnosis. The contribution of this work is not in proposing new UQ algorithms, but rather in systematically applying existing UQ techniques to PCG-based diagnosis. The results show that uncertainty estimation gives new insights into model behavior beyond performance metrics, which is crucial in clinical applications.

Limitations and future work

Despite the promising results of the proposed UQ framework, some limitations need to be acknowledged. The study is conducted on standard benchmark PCG data without patient-wise splitting or multi-institutional evaluation. Although this allows a fair comparison with existing studies, more extensive validation in an independent patient group is required to further assess generalizability. The robustness analysis relies on synthetic noise perturbations, which approximate but may not sufficiently model clinical complexity in real-time settings. While uncertainty quantification is informative about the reliability of predictions, the practical implementation of uncertainty-aware PCG analysis in real-time decision support systems in clinical settings is still an important topic to explore. Overcoming these challenges will require using larger, multi-center datasets and a closer working relationship between engineers and clinicians.

As future work, further research will focus on evaluating the proposed framework using patient-independent protocols and datasets acquired across multiple healthcare centers, incorporating uncertainty-aware decision thresholds for automatic referral systems, and investigating lightweight UQ techniques for real-time auscultation devices. Also, expanding the model to incorporate feedback from clinicians and longitudinal patient data might improve its flexibility and clinical relevance. Integrating such information could enhance trustworthiness and facilitate safer clinical decision-making.

Conclusion

Recent advances in deep learning have greatly advanced the automatic analysis of biomedical signals, but the trustworthiness and clinical adoption of such systems rely not only on high predictive performance but also on their capacity to reason and communicate uncertainty. In this paper, we proposed an uncertainty-aware framework for PCG-based heart valve disease classification. The proposed method achieves state-of-the-art performance on the HVD dataset while providing explicit predictive uncertainty estimates via several uncertainty quantification methods. Comprehensive experimental studies on clean and noisy data establish that the framework preserves robust performance in the face of realistic degradation, and further show that the resulting uncertainty behavior correlates well with higher calibration and a greater degree of uncertainty for less confident or low-quality inputs. To the best of our knowledge, this work is among the first attempts to systematically incorporate UQ for PCG based heart valve disease analysis and overcomes a crucial drawback of previously proposed deterministic approaches. By accompanying the predictions with confidence (certain and correctly classified) estimates, the method increases clinical trust and supports risk-aware decision-making, thereby promoting progress toward more reliable and safe PCG-based diagnostic systems for in-field use.

Author contributions

KP-S: conceptualization, methodology, software, validation, formal analysis and investigation, original draft writing, and visualization. M-N: conceptualization, methodology, review and editing of the writing, formal analysis and investigation, and supervision. RA-U: validation, writing review and editing, and supervision. KS-S: conceptualization, supervision, formal analysis, writing review, and editing. All authors reviewed the manuscript.

Funding

Open access funding provided by Amrita Vishwa Vidyapeetham. No funding.

Data Availability

The dataset analysed during the current study are publicly available in the GitHub repository, https://github.com/yaseen21khan/Classification-of-Heart-Sound-Signal-Using-Multiple-Features-

Declarations

Competing interests

The authors declare no competing interests.