Efficiency, Sensitivity and Specificity of Automated Auscultation Diagnosis Device for Detection and Discrimination of Cardiac Murmurs in Children

Abstract

Objective

Intelligent electronic stethoscopes and computer-aided auscultation systems have highlighted a new era in cardiac auscultation in children. Several collaborative multidisciplinary researches in this field are performed by physicians and computer specialists. Recently, a novel medical software device, Automated Auscultation Diagnosis Device (AADD), has been reported with intelligent diagnosing ability to differentiate cardiac murmur from breath sounds in children with normal and abnormal hearts due to congenital heart disease. The aim of this study is to determine efficiency, sensitivity and specificity of the diagnoses made by this AADD in children with and without cardiac disease.

Methods

We performed a cross-sectional study to determine efficiency, sensitivity and specificity of diagnoses made by AADD. Our patient population was two groups of children with and without cardiac disease(563 patients and 50 normal). SPSS version 16 was used to calculate sensitivity, specificity and efficiency and descriptive analysis.

Findings

Using cardiac sound recording in four conventional cardiac areas of auscultation (including aortic, pulmonary, tricuspid and mitral), AADD proved to have a ≥90% sensitivity, specificity and efficiency for making the correct diagnosis in children with heart disease and 100% diagnostic accuracy in children with normal hearts either with or without innocent murmurs.

Conclusion

Considering the high sensitivity, specificity and efficiency of AADD for making the correct diagnosis, application of this software is recommended for family physicians to enhance proper and timely patients’ referral to pediatric cardiologists in order to provide better diagnostic facilities for pediatric patients who live in deprived and underserved rural areas with lack access to pediatric cardiologists.

Introduction

Congenital heart diseases affect approximately 0.8% of newborn infants and account for about 10% of all congenital anomalies. Intelligent electronics stethoscope has highlighted a new era in computer-aided cardiac auscultation and automatic diagnosis. However, published reports regarding the efficiency, sensitivity and specificity of these types of computer-aided auscultation devices are few^[1–3]. The aim of this study is to determine the diagnostic accuracy and sensitivity, specificity and efficiency of one of these softwares which is called Automated Auscultation Diagnosis Device(AADD).

Subjects and Methods

Software characteristics

The software is called Automated Auscultation Diagnosis Device(AADD) for cardiac murmurs in children It is based on Arash-Band which is a method for characterizing heart murmurs based on the heart structures producing them (patent publication number: US 2011/0021939 A1, Iran patent no. 1390/02/25-69899). Arash-Band method provides a technique by which a discriminative frequency band is attributed for each heart disease. The spectral energies of the discriminative bands are employed to constitute the feature vectors for classification. Congenital heart diseases screening is performed by classifying the feature vectors using a multi-layer perceptron (MLP) neural network^{[4, 5]}.

The diagnosis method of the AADD relies on classification of congenital heart diseases based on the heart sections that cause the pathological murmurs. Children's heart sounds with congenital diseases show energy concentrations over specific frequency bands which are different from those of healthy children^{[4, 5]}. We call these bands Arash-Bands. The spectral energies of the Arash-Bands can be used as a feature vector for screening congenital heart diseases in children. The final disease-screening task is performed by a neural network that acts upon a feature vector constructed from the energy contents of the Arash-Bands for all the diseases of interest.

Fig. 1 shows the main panel of this intelligent system which provides several options for the physicians. The AADD is also an advanced phonocardiogram. The physician can save the information of each patient in (patient information panel). Moreover, it enables the physician to delete, view or listen to recorded sound signals (in sound library panel).One of the advantages of this system is that it provides the possibility of recording sound signals from different locations (ULSB, URSB, LLSB, and APAX). The physicians can also add their comments online. The AADD automatically records from each sound record location (aortic, pulmonary, mitral and tricuspid areas) for 10 seconds. Although it also enables the physicians to set their preferred recording time, our studies indicate that analyzing of sound signals is best performed after a 10 second-record. The analysis of each record is reported separately.

Fig. 1.

Main panel of Automated Auscultation Diagnosis Device for cardiac murmurs in children (AADD). Heart sounds signals are recorded using a wireless or normal electronic stethoscope. The system shows the diagnosing results, segmentations and other characteristics of the signals in related boxes

Besides automatic diagnosis the system is an advanced digital phonocardiogram. There are also different options on the system such as, listening and payback of recorded sounds with adjustable intensity levels, various selectable times scales for viewing the whole or part of a sound cycle and extra. In addition, the system uses an intelligent and unique method for automatic segmentation.

That is, using spectral properties of pediatric heart sounds and the influences of respiration on systolic and diastolic timing of cardiac cycles^{[6, 7]}. The AADD automatically analysis the heart sound signals but, it gives physician many options and possibilities of using the system as a computer-aided auscultation device. For instance, viewing presence of additional murmur in systolic and diastolic intervals can be observed by setting the relative soft bottom. Fig. 3 shows a depicted sound signal related to a child with VSD disease. As it is well seen on Figs. 2 and 3, it is possible to open different overlapped windows for diagnosing of the signals simultaneously and this is a special advantage of the AADD. We have chosen only two overlapped diagnosis windows for simplicity of the explanation. The additional murmur in systolic interval is highlighted. Besides automated diagnosis, the AADD provides many heart sounds processing options including graphical interpretation of the recorded auscultation signals as indicated in Figs. 2 and 3.

Fig. 2.

One cycle of the recorded sound signal of a normal subject in which the first sound (S1) and the second sound (S2) are separated automatically by the system. The red dots show end and start of each sound, by pressing the pink dot at end of the cycle, physician can view next cycle

Fig. 3.

One cycle of the recorded sound signal of a VSD subject. An additional murmur in systolic interval is shown in yellow color

Study design and study population

We performed this cross-sectional study to determine precise efficiency, sensitivity and specificity of the diagnoses made by the novel Automated Auscultation Diagnosis Device for cardiac murmurs in children (AADD). Ethical approval was obtained from the Hospital Ethical

Committee. The study was done in two phases. In the initial phase, we made a recorded audio data bank of cardiac auscultation of 563 children aged 2 to 12 years with respective precise echocardiography diagnosis as shown in Table 1.

Table 1.

Characteristic of recorded cardiac auscultation of 563 children aged 2 to 12 years

Final Diagnosis Decision	Patients Cases	Mean age (years)
No murmur – normal heart	71	8.3
Innocent murmur	213	7.65
Ventricular septal defect	94	5.72
Mitral regurgitation	43	5.64
Atrial septal defect	37	7.12
Bicuspid aortic valve	28	8.5
Aortic stenosis	27	9.21
Tetralogy of Fallot	19	6.34
Patent ductus arteriosus	17	7.54
Aortic regurgitation	14	8.1

All descriptive analysis was performed using SPSS 16.Using AADD, we analyzed the recorded sounds. The diagnoses made by AADD were compared with echocardiography diagnoses performed by four skilled pediatric cardiologists. Specificity, sensitivity and efficiency of results obtained by AADD were calculated according to the following formulas:

In the second phase, 50 normal volunteer children of elementary schools were entered into the study. Fifteen of them had no cardiac murmur and 35 of them had innocent murmurs. After obtaining informed consent of their parents and obtaining the children's permission, all of the 50 children underwent comprehensive echocardiography examination by the pediatric cardiologist. Using the echocardiographic diagnosis performed by pediatric cardiologists as the gold standard, the diagnostic accuracy of pediatricians and AADD were compared (Table 2).

Table 2.

Correctness of diagnoses made by the medical device versus average diagnoses made by 4 pediatricians

	Correct diagnosis	Wrong diagnosis	Total number
Pediatricians	40	10	50
Medical Device	50	0	50

Findings

As it is indicated in Fig. 4, specificity, sensitivity and efficiency of the Automated Auscultation Diagnosis Device are 97%, 90% and 93% respectively. As it is shown in Table 2, specificity of AADD diagnoses and pediatricians’ diagnoses was 100% versus 80%.

Fig. 4.

Indicates efficiency, sensitivity and specificity of the software device. Diagnosed results obtained under two circumstances, single-area recoded audio file and four-area recorded audio file, confirmed by echocardiography diagnoses. The terms single and four area were applied according to the number of four auscultation areas (tricuspid, mitral, aortic and pulmonic areas), used for recording. In both phases of the study, echocardiographic diagnosis was made by pediatric cardiologists as the gold standard for comparison

Discussion

Several recent studies have reported the efficacy of computer-assisted cardiac auscultation^{[8, 9]}, however, as to the best of our knowledge few studies study have been performed to report the efficiency of such systems to detect cardiac murmurs in children^[10–15]. Pretorius et al reported a novel method for computer-aided auscultation. Their study included children aged 5±1 years with and without heart disease^[10]. They reported a sensitivity of 91% and a specificity of 94%. Visagie et al reported a 84% sensitivity and a 86% specificity for in adults. They recorded the auscultatory data simultaneously in 31 healthy individuals and in 21 patients. They did not study murmurs^[11].

According to Johns Hopkins University School of Medicine Studies, average specificity of board-certified primary care physician is 80% in USA^[12]. Our study showed reliable and high efficiency, sensitivity and specificity of the novel software device for screening and automatic diagnosis of children heart disease. In health organizations that are based on family physician referral structures, use of this software device can add to the accuracy of diagnoses and increase in number of timely and appropriate referrals. This medical software device can also be used in rural and deprived areas where highly skilled physicians are not widely available.

Limitations and Applications: Since sensitivity, specificity and efficiency were not measured in the second phase of study, this can be considered as a limitation in the methodology of this study.

Conclusion

We indicated significantly high efficiency, sensitivity and specificity of a novel medical device for automatic detection of cardiac murmurs in children with and without heart disease. AADD, can be particularly a promising tool to increase the number of appropriate and timely referrals by family physicians to pediatric cardiologists in deprived and underserved rural areas. We expect that by completing the AADD ausultatory data bank, the diagnostic accuracy, sensitivity, specificity and efficiency be increased to 100%.

Conflict of Interest

None