Abstract
Aims
Computer-assisted auscultation has become available to assist clinicians with physical examinations to detect congenital heart disease (CHD). However, its accuracy and effectiveness remain to be evaluated. This study seeks to evaluate the accuracy of auscultations of abnormal heart sounds of an artificial intelligence-assisted auscultation (AI-AA) platform we create.
Methods and results
Initially, 1397 patients with CHD were enrolled in the study. The samples of their heart sounds were recorded and uploaded to the platform using a digital stethoscope. By the platform, both remote auscultation by a team of experienced cardiologists from Shanghai Children’s Medical Center and automatic auscultation of the heart sound samples were conducted. Samples of 35 patients were deemed unsuitable for the analysis; therefore, the remaining samples from 1362 patients (mean age—2.4 ± 3.1 years and 46% female) were analysed. Sensitivity, specificity, and accuracy were calculated for remote auscultation compared to experts’ face-to-face auscultation and for artificial intelligence automatic auscultation compared to experts’ face-to-face auscultation. Kappa coefficients were measured. Compared to face-to-face auscultation, remote auscultation detected abnormal heart sound with 98% sensitivity, 91% specificity, 97% accuracy, and kappa coefficient 0.87. AI-AA demonstrated 97% sensitivity, 89% specificity, 96% accuracy, and kappa coefficient 0.84.
Conclusions
The remote auscultations and automatic auscultations, using the AI-AA platform, reported high auscultation accuracy in detecting abnormal heart sound and showed excellent concordance to experts’ face-to-face auscultation. Hence, the platform may provide a feasible way to screen and detect CHD.
Keywords: Artificial intelligence-assisted auscultation, Remote auscultation, Heart murmur, Congenital heart disease
Introduction
Congenital heart disease (CHD) is one of the most common birth defects and is a leading cause of death among infants.1 Approximately one or two of every 1000 newly born babies have critical CHDs and require invasive intervention in the neonatal period.2 Without timely diagnosis and treatment, approximately one-third of children with CHD would die within the first year after birth. Therefore, accurate and timely detection of CHD is crucial. A combination of cardiac auscultation and oxygen saturation has been reported as an essential and effective procedure for CHD screening.2 The geographical distribution of medical resources are notably unbalanced in China.3 In some areas of China, a heart murmur may be the first and only intimation for the detection of CHD.4 It usually requires sufficient training for some primary care physicians to identify a heart murmur accurately. The training process is time-consuming and involves a lot of effort and resources.5 Failing to accurately discern a heart murmur can result in a delayed diagnosis and treatment of CHD.6 The problem is particularly prominent given the lack of medical resources and personnel such as specialized paediatric cardiologists in some remote rural areas of China.
Recently, computer-assisted auscultation has become available to assist physicians with physical examination of the heart.4,7,8 It allows experienced cardiologists to evaluate heart sound samples that are collected and transmitted from geographically remote areas. Hence, children located in such areas can be timely diagnosed by experienced cardiologists. Additionally, the computer can automatically analyse the uploaded samples and distinguish between the abnormal and normal heart sounds and then provide a rapid diagnosis. A significant amount of research has been carried out to evaluate the accuracy of computer-aided auscultation. Developments of signal processing and classification algorithms can be seen, which were designed to analyse heart murmurs.5 Despite all the efforts, the adoption of this technology in clinical practice is not widespread; this is due to the fact that the accuracy and effectiveness of computer-assisted auscultation remain to be rigorously evaluated.4,6,8
Therefore, the study aimed to (i) establish an effective screening platform and assess the sensitivity and specificity of remote auscultation and automatic auscultation in identifying abnormal heart sounds through the artificial intelligence (AI)-assisted auscultation (AI-AA) platform and (ii) evaluate the feasibility of the AI-AA platform we established.
Methods
Study design and population
This study was conducted in compliance with the Declaration of Helsinki and was approved by the ethics committee of the Shanghai Children’s Medical Center (SCMCCIRB-K2020039-1). Oral informed consents were obtained from the parents of participants. We enrolled 1397 children diagnosed with CHD, who were admitted for surgery in the Shanghai Children’s Medical Center. Echocardiography of all the patients were performed. We excluded children who were previously operated for CHD. The flow chart of this study is shown in Figure 1.
Figure 1.
The flow chart of this study.
Baseline characteristics
Data on characteristics of patients, including medical histories and diagnosis, were obtained from the electronic medical record system of our centre.
AI-AA platform and audio recordings
The AI-AA platform was established using which both experienced cardiologists and AI could auscultate the uploaded samples and perform a diagnosis. The platform consisted of a collection of heart sounds and an auscultation system. It operated with a digital stethoscope, laptop, and software application. Heart sound samples were collected with a digital stethoscope (3M™ Littmann® Electronic Stethoscope Model 3200, MN, USA). They were then transferred via Bluetooth to a laptop and uploaded to the AI-AA platform for remote auscultation by experts based at a national cardiac centre. Heart sounds, each of 10 s interval, were recorded from four main auscultation areas—pulmonary valve, aortic valve, mitral valve, and tricuspid. All the recordings and echocardiograms were taken on the same day. After the heart sounds were uploaded and transmitted to the platform, the experienced cardiologists could remotely auscultate the samples in time or immediately if needed and label them by logging in to the platform using a computer. Additionally, AI could provide automatic auscultation in approximately 5 s and display a diagnosis of either normal or abnormal heart sounds on the platform. The analysis of each record was separately reported.
Echocardiography
All the patients were diagnosed by preoperative echocardiography. Some patients also underwent cardiac catheterization.
Processing of heart sounds
A convolutional neural network (CNN) is trained to classify heart sounds. We randomly divide the total 3000 heart sound files into five folds of equal size to perform five-fold cross-validation. Each heart sound file is 10-s long and recorded with a 4 kHz sampling rate. They are firstly segmented into 2-s frames with 1-s overlap. Each frame is then band-pass filtered to 16 sub-bands using Gammatone filter banks. We use Hilbert transform to extract one-dimensional envelope features of size 32 for each sub-band. Next, the envelop features of 16 sub-bands are stacked to form a two-dimensional, time-frequency, and image-like feature representation. Therefore, each frame is transformed into a two-dimensional image as input of CNN. The network consists of two convolutional layers, a max-pooling layer, and three fully connected layers. The kernel size of convolutional layers is 3, the number of both output channels is 32, and stride is set to 1. The stride of the pooling layer is 2. The output sizes of three fully connected layers are 64, 16, and 2, respectively. The activation function is ReLU. The batch size is set to 128. The loss function is binary cross-entropy loss because there are two classes: normal and pathological. At each iteration, the network adjusted the model parameters using backpropagation to minimize the difference between the predicted probability and the labelled ground-truth class. The resulting probabilities of the frames were averaged as the final output file-level prediction.
Auscultation of cardiologists
Face-to-face auscultations were performed by experienced in-house cardiologists and their opinion were considered as the gold standard to detect a heart murmur here. Both remote and AI auscultation by the platform were compared to it.
Two experienced cardiologists performed remote auscultations on the AI-AA platform and labelled abnormal heart sounds. The third cardiologist was consulted to overcome any uncertainties. Different cardiologists were chosen to perform remote and face-to-face auscultations. All the cardiologists were unaware of the echocardiographic findings while performing auscultations.
Statistical analysis
The sensitivity, specificity, and accuracy of remote and AI automatic auscultation, to detect abnormal heart sounds using the AI-AA platform, were calculated. Statistical analyses were performed using SPSS software version 21.0 (IBM Corp). Continuous values were expressed as mean and standard deviation, whereas categorical values were expressed as absolute numbers and percentages. Categorical variables were compared using the χ2 test or the Fisher’s exact test.
The concordance between different approaches was assessed by analysing the kappa (κ) coefficient. κ coefficients of >0.8 were considered to represent excellent agreement.
Results
Samples of 35 subjects were removed due to—loud crying noise or unsuccessful collection. Hence, the heart sounds samples of 1362 patients were eligible for the analyses. The patients (626 females, 46%) with a mean age of 2.4 years (standard deviation—3.1, median—0.9, and age range—1 day to 15.9 years), were enrolled. Face-to-face auscultation diagnosed 1194 patients with abnormal heart sound and 168 with normal heart sound. Cardiac diagnoses determined by echocardiography are summarized in Table 1.
Table 1.
Cardiac diagnoses determined by echocardiography
| Diagnosis | Number (%) |
|---|---|
| Ventricular septal defect (VSD) | 584 (43) |
| Atrial septal defect (ASD) | 239 (18) |
| Tetralogy of Fallot (TOF) | 180 (13) |
| Pulmonary atresia (PA) | 85 (6) |
| Coarctation of the aorta (COA) | 39 (3) |
| Double outlet right ventricle (DORV) | 37 (3) |
| Aortic valve stenosis (AS) | 32 (2) |
| Atrioventricular septal defect (AVSD) | 32 (2) |
| Mitral insufficiency (MR) | 32 (2) |
| Patent ductus arteriosus (PDA) | 31 (2) |
| Pulmonary stenosis (PS) | 22 (2) |
| Total anomalous pulmonary valve connection (TAPVC) | 19 (1) |
| Ebstein’s anomaly | 18 (1) |
| Coronary origin from pulmonary artery | 12 (1) |
The results of remote auscultations by cardiologists using the AI-AA platform were compared with the face-to-face auscultations. The results showed that remote auscultations by cardiologists detected abnormal heart sounds with a sensitivity, specificity and accuracy of 98% [95% confidence interval (CI): 97–99%], 91% (95% CI: 87–95%), and 97% (95% CI: 96–98%), respectively. The k coefficient was found to be 0.87 (95% CI: 0.83–0.91). Please see Table 2.
Table 2.
Cardiologists’ remote auscultation compared to face-to-face auscultation
| Remote auscultation | Face-to-face auscultation |
||
|---|---|---|---|
| Abnormal heart sound | Normal heart sound | Total | |
| Abnormal heart sound | 1171 | 15 | 1186 |
| Normal heart sound | 23 | 153 | 176 |
| Total | 1194 | 168 | 1362 |
Sensitivity, specificity, accuracy, and k coefficient are calculated. Sensitivity of 98% (1171 of 1194; 95% CI: 97–99%), specificity of 91% (153 of 168; 95% CI: 87–95%), accuracy of 97% (1324 of 1362; 95% CI: 96–98%), and k coefficient of 0.87 (95% CI: 0.83–0.91).
The AI automated auscultations were compared with the face-to-face auscultations. The results showed that the AI automatic auscultations detected abnormal heart sounds with a sensitivity, specificity, and accuracy of 97% (95% CI: 96–98%), 89% (95% CI: 84–94%), and 96% (95% CI: 95–97%), respectively. The k coefficient was found to be 0.84 (95% CI: 0.79–0.88). More details can be found in Table 3.
Table 3.
AI automatic auscultation compared to face-to-face auscultation
| AI | Face-to-face auscultation |
||
|---|---|---|---|
| Abnormal heart sound | Normal heart sound | Total | |
| Abnormal heart sound | 1162 | 18 | 1180 |
| Normal heart sound | 32 | 150 | 182 |
| Total | 1194 | 168 | 1362 |
Sensitivity, specificity, and k coefficient are calculated. Sensitivity of 97% (1162 of 1194; 95% CI: 96–98%), specificity of 89% (150 of 168; 95% CI: 84–94%), accuracy of 96% (1312 of 1362; 95% CI: 95–97%), and k coefficient of 0.84 (95% CI: 0.79–0.88).
Additionally, we analysed the characteristics of phonocardiograms of the heart sounds using AI. The representative phonocardiograms of ventricular septal defect and mitral regurgitation patients with AI diagnosis are shown in Figure 2. Each phonocardiogram is presented as both the original and the wavelet transformed waveforms. Heart murmurs are usually observed between the first (S1) and second (S2) heart sounds.
Figure 2.
Representative phonocardiograms of AI diagnosis compared to face-to-face auscultation. (A) Representative phonocardiogram of a VSD patient. (B) Representative phonocardiogram of a MR patient. Each phonocardiogram is presented as two waveforms (a) original waveform of the heart sound and (b) waveform after wavelet transform. Heart murmurs are located between the first (S1) and the second (S2) heart sounds.
Discussion
We constructed the AI-AA platform and we evaluated the detection accuracy of abnormal heart sounds in CHD patients that were admitted for surgery, through remote auscultations by experienced cardiologists and AI. We compared both the remote auscultation and AI auscultation with face-to-face auscultations by experienced cardiologists. We found that both cardiologists’ remote auscultation and AI automatic auscultation using the platform had a higher sensitivity and specificity, and showed excellent agreement with the face-to-face auscultations.
Traditionally, a stethoscope is used for auscultations of heart sounds. The technique largely depends upon clinical expertise and extensive hearing training. It is a crucial step for the early diagnosis of CHD. Despite being low-cost, traditional auscultation are limited in low-resource settings with a shortage of trained cardiologists.9 In our country, due to the lack of sufficient auscultation abilities of primary physicians located in some rural areas, children with CHD from such areas often remain undetected. It results in a delayed diagnosis and treatment. Previous study found that the missed diagnosis rate of severe CHD before discharged from birth hospital was 71%.10 Therefore, it is crucial to screen for CHD in early neonatal period. The emergence and development of electronic stethoscopes facilitate better remote diagnoses and hence timely detection of CHD in rural areas.11,12 Previous studies have shown that remote auscultation is cost-effective for CHD screening and provides a reliable diagnosis.11,13 However, their results were limited by a small sample size. In this study, heart sounds of 1397 patients were recorded using the digital stethoscope and uploaded to the established AI-AA platform. We found that remote auscultations can detect abnormal heart sounds with a high accuracy of 97% when compared with traditional face-to-face auscultations. There was a close agreement between remote and face-to-face auscultations, which suggests that remote auscultations by cardiologists via AI-AA platform can achieve results similar to face-to-face auscultations. Timely and accurate detection of heart murmurs of CHD patients in remote areas, where medical resources are poor and limited, can be achieved via AI-AA platform. In this case, children were diagnosed timely and accurately, thus reducing delayed treatment. A large amount of time and training costs for conventional detection of CHD can be saved in remote areas using the established AI-AA platform. The local paediatricians only need a digital stethoscope and iPad to collect and upload heart sounds.
There has been significant increase interest in using AI technology for cardiac auscultation to detect CHD. AI algorithms have made great progress in detecting heart murmurs.5,6,14–16 However, multiple clinical validations are required to confirm their accuracy before recommending their clinical use. Previous studies have verified the accuracy of AI diagnoses. However, their sample size was small.4,7,17 In this study, heart sounds of 1362 patients were recorded, which is by far the largest sample size as we know. We compared auscultations by AI with face-to-face auscultations by experienced cardiologists and found that AI auscultation correctly identified abnormal heart sounds with 97% sensitivity and 89% specificity. AI auscultations have high sensitivity and specificity for the detection of abnormal heart sounds, similar to the levels reported by face-to-face auscultations. It makes AI auscultation a potentially useful screening tool for CHD. Compared to the conventional algorithms, our method uses CNN to learn the feature representation. CNN can improve the accuracy of the diagnosis using massive data and continuous training. Moreover, the designed convolutional network takes advantage of local spatial coherence and has fewer weights, as some parameters are shared, making it a better choice for feature extraction.6,18 Even though the recordings were made in a busy clinical environment, the algorithm could filter out the noise and detect abnormal heart sounds with high accuracy similar to that of a cardiologist, indicating its potential clinical feasibility. Fifty (3.6%) recordings were misdiagnosed. The reasons for these misdiagnoses include the presence of the low-amplitude waves and poor sound quality. For these recordings, as both cardiologists and AI can continuously perform auscultations and label the results on the platform, the algorithm can be continuously trained and optimized to obtain better auscultation accuracy in the future.
Limitations
The findings shed light on the high auscultation accuracy and feasibility of the AI-AA platform for the detection of CHD. However, there are some limitations. First, all the patients were diagnosed with CHD and prepared for heart surgery. This may cause some bias. For this reason, we keep the cardiologists blind to the diagnosis of the patients to reduce bias. Second, due to the high rate of missed diagnosis in some areas of our country, many children cannot be diagnosed and treated in time, which has caused a heavy burden to the society. Therefore, the platform aims to detect abnormal heart sounds without distinguishing between innocent and pathological heart murmurs. This may cause a small amount of waste of resources. Considering the high detection rate and accuracy of the platform, we think it is worthwhile.
Conclusion
In this study, we established the AI-AA platform and demonstrated that it is highly accurate in detecting abnormal heart sound and showed excellent agreement with experts’ face-to-face auscultation. It has great potential to be used for the screening of CHD in children from remote areas where the lack of skill and technology hinders timely detection of CHD. Early detection of CHD in children from remote areas can be possible with adoption of the AI-AA platform.
Acknowledgements
We would like to thank Editage (www.editage.com) for English language editing, Shanghai FitGreat Network Technology Corporation for coordinating the study and Jihong Huang for statistical consultation.
Funding
This study was supported by the Shanghai Science and Technology Committee (19441904400), Shanghai Municipal Commission of Economy and Informatization (Shanghai Artificial Intelligence Pilot Application Scenario), National Natural Science Foundation of China (81970267 and 91670464), Shanghai Municipal Health Commission (2019SY046), Science and Technology Commission of Shanghai Municipality (20025800300), and Shanghai Jiaotong University School of Medicine (ZH2018ZDA24 and DLY201815).
Data availability statement
The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.
Conflict of interest: none declared.
References
- 1. Benjamin EJ, Muntner P, Alonso A, Bittencourt MS, Callaway CW, Carson AP, Chamberlain AM, Chang AR, Cheng S, Das SR, Delling FN, Djousse L, Elkind MSV, Ferguson JF, Fornage M, Jordan LC, Khan SS, Kissela BM, Knutson KL, Kwan TW, Lackland DT, Lewis TT, Lichtman JH, Longenecker CT, Loop MS, Lutsey PL, Martin SS, Matsushita K, Moran AE, Mussolino ME, O'Flaherty M,, Pandey A,, Perak AM, Rosamond WD, Roth GA, Sampson UKA, Satou GM,, Schroeder EB, Shah SH, Spartano NL, Stokes A, Tirschwell DL, Tsao CW, Turakhia MP, VanWagner LB, Wilkins JT,, Wong SS, Virani SS;American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics-2019 update: a report from the American Heart Association. Circulation 2019;139:e56–e528. [DOI] [PubMed] [Google Scholar]
- 2. Zhao QM, Ma XJ, Ge XL, Liu F, Yan WL, Wu L, Ye M, Liang XC, Zhang J, Gao Y, Jia B, Huang GY;Neonatal Congenital Heart Disease screening group. Pulse oximetry with clinical assessment to screen for congenital heart disease in neonates in China: a prospective study. Lancet 2014;384:747–754. [DOI] [PubMed] [Google Scholar]
- 3. He Y, Xu W, Su Z, Liu K,, Zhang H.. Addressing the rising burden of congenital heart disease in China. Lancet Child Adolesc Health 2020;4:e7. [DOI] [PubMed] [Google Scholar]
- 4. Lee C, Rankin KN, Zuo KJ,, Mackie AS.. Computer-aided auscultation of murmurs in children: evaluation of commercially available software. Cardiol Young 2016;26:1359–1364. [DOI] [PubMed] [Google Scholar]
- 5. Ahmad MS, Mir J, Ullah MO, Shahid M,, Syed MA.. An efficient heart murmur recognition and cardiovascular disorders classification system. Australas Phys Eng Sci Med 2019;42:733–743. [DOI] [PubMed] [Google Scholar]
- 6. Kang S, Doroshow R, McConnaughey J,, Shekhar R.. Automated identification of innocent still's murmur in children. IEEE Trans Biomed Eng 2017;64:1326–1334. [DOI] [PubMed] [Google Scholar]
- 7. Pretorius E, Cronje ML,, Strydom O.. Development of a pediatric cardiac computer aided auscultation decision support system. Conf Proc IEEE Eng Med Biol Soc 2010;2010:6078–6082. [DOI] [PubMed] [Google Scholar]
- 8. Thompson WR, Reinisch AJ, Unterberger MJ,, Schriefl AJ.. Artificial intelligence-assisted auscultation of heart murmurs: validation by virtual clinical trial. Pediatr Cardiol 2019;40:623–629. [DOI] [PubMed] [Google Scholar]
- 9. Leng S, Tan RS, Chai KT, Wang C, Ghista D,, Zhong L.. The electronic stethoscope. Biomed Eng Online 2015;14:66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Ma X-J,, Huang G-Y.. Current status of screening, diagnosis, and treatment of neonatal congenital heart disease in China. World J Pediatr 2018;14:313–314. [DOI] [PubMed] [Google Scholar]
- 11. Pyles L, Hemmati P, Pan J, Yu X, Liu K, Wang J, Tsakistos A, Zheleva B, Shao W,, Ni Q.. Initial field test of a cloud-based cardiac auscultation system to determine murmur etiology in rural China. Pediatr Cardiol 2017;38:656–662. [DOI] [PubMed] [Google Scholar]
- 12. Lakhe A, Sodhi I, Warrier J,, Sinha V.. Development of digital stethoscope for telemedicine. J Med Eng Technol 2016;40:20–24. [DOI] [PubMed] [Google Scholar]
- 13. Finley JP, Warren AE, Sharratt GP,, Amit M.. Assessing children's heart sounds at a distance with digital recordings. Pediatrics 2006;118:2322–2325. [DOI] [PubMed] [Google Scholar]
- 14. Andrisevic N, Ejaz K, Rios-Gutierrez F, Alba-Flores R, Nordehn G,, Burns S.. Detection of heart murmurs using wavelet analysis and artificial neural networks. J Biomech Eng 2005;127:899–904. [DOI] [PubMed] [Google Scholar]
- 15. El-Segaier M, Lilja O, Lukkarinen S, Sornmo L, Sepponen R,, Pesonen E.. Computer-based detection and analysis of heart sound and murmur. Ann Biomed Eng 2005;33:937–942. [DOI] [PubMed] [Google Scholar]
- 16. Zuhlke L, Myer L,, Mayosi BM.. The promise of computer-assisted auscultation in screening for structural heart disease and clinical teaching. Cardiovasc J Afr 2012;23:405–408. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Grgic-Mustafic R, Baik-Schneditz N, Schwaberger B, Mileder L, Binder-Heschl C, Pansy J, Koestenberger M, Urlesberger B, Avian A,, Pichler G.. Novel algorithm to screen for heart murmurs using computer-aided auscultation in neonates: a prospective single center pilot observational study. Minerva Pediatr 2019;71:221–228. [DOI] [PubMed] [Google Scholar]
- 18. Kocharian A, Sepehri AA, Janani A,, Malakan-Rad E.. Efficiency, sensitivity and specificity of automated auscultation diagnosis device for detection and discrimination of cardiac murmurs in children. Iran J Pediatr 2013;23:445–450. [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.
Conflict of interest: none declared.