Effectiveness of Teaching Cardiac Auscultation to Residents During an Elective Pediatric Cardiology Rotation

Abstract

The purpose of the current study was to assess the effectiveness of randomized tracks of pre-recorded cardiac sounds as a teaching tool for cardiac auscultation with the emphasis on recognizing murmurs when present, distinguish functional from organic murmurs, and detect heart disease by auscultation. At pre-test and post-test 26 residents listened to 15 randomized tracks of live-recorded cardiac sounds and identified key features. Results indicate residents improved at detecting any murmur (66% vs 76%, p = 0.007) and functional murmur (37% vs 54%, p = 0.048), and marginally improved at detecting organic murmur (75% vs 84%, p = 0.129). Detection of absence of murmur declined slightly (69% vs 62%, p = 0.723). The post-test difference in identifying organic vs functional murmurs was striking (84% vs 54%, p < 0.001). Detecting heart disease (sensitivity) improved significantly (76% to 86%, p = 0.016), but there was scant improvement in detecting no disease (specificity) (55% vs 59%, p = 0.601). Residents increased their ability to detect heart disease when present. However, the false positive rate for diagnosis of heart disease remained quite high. To ensure that appropriate referrals will be made, teaching should specifically target the confident recognition of functional murmurs.

Introduction

Cardiac auscultation, a potentially critical component of the clinical assessment of the cardiovascular system, has been neglected because non-invasive tests such as the cardiac ultrasound have become readily available world wide. [11] Among non-cardiologists the skills of cardiac auscultation have long been reported as poor: 523 physicians attending a general medical conference listened to 15 live recorded cardiac sounds and misidentified half; by contrast 410 physicians attending a national cardiology conference misidentified only one third. [1] Five-hundred forty-one medical students and residents in training correctly recognized only 20% of recorded sounds and murmurs, [6] and pediatrics residents who listened to simulated sounds and murmurs had overall diagnostic accuracy of only 33%. [2] Despite the bleak prospects suggested by these findings, it is still generally agreed that primary-care physicians should be taught basic cardiac auscultation at least sufficient to screen healthy children's cardiac murmurs, and recognize disease when present. [11] This would ensure appropriate referrals to subspecialists. However, to our knowledge teaching and testing of these general aspects of pediatric murmur (present/absent, functional/organic) has not been emphasized, but rather the focus has been placed on recognition of the type of murmur (e.g., systolic, diastolic, continuous). [1] These general aspects are important because they directly support the assessment of heart disease as likely absent or present.

In 1999 the Accreditation Council for Graduate Medical Education (ACGME) began requiring residency programs to write detailed goals and standards, and to provide standardized tests of performance. [4] As one part of our pediatric cardiology faculty's response to the ACGME's requirements, we revised our teaching and testing of the assessment of murmurs. The purpose of the current study was, therefore, to determine how well residents attending our one-month pediatric cardiology rotation would learn to (a) identify murmur when present, (b) distinguish functional from organic murmurs, and (c) diagnose heart disease when present by auscultation alone.

Materials and Methods

This was a single-group repeated-measures (pre-test/post-test) educational intervention incorporating alternate forms of the performance test conducted from 2002 to 2005.

Subject

Second- or third-year pediatrics or medicine/pediatrics residents attended a 4-wk elective in pediatric cardiology and took part in the study, which was approved by the Institutional Review Board as educational research.

Testing

At the beginning and end of the rotation each resident listened to and scored 15 CD tracks of live recordings of cardiac sounds and murmurs. The recordings were obtained from infants and children over a broad age range using an electronic stethoscope (SimulScope 718-7005, Cardionics Inc., Houston, TX). The sounds were digitized and transferred to a CD format. Although this process altered somewhat the physical characteristics of the recorded sounds, the overall quality remained excellent, and the respiratory phases were easily recognizable for detection of S2 splitting.

Table 1 shows the distribution of murmurs and heart disease among the 15 tracks. In preparing these recordings, the assessment of functional vs. organic murmur was made by a pediatric cardiologist on clinical grounds. [8] Heart disease was confirmed by echocardiography. We sought to obtain a reference standard by spectral analysis of the recorded sounds, but with the exception of a few tracks, background noise prevented meaningful interpretation. In the absence of this gold standard, all scores of the senior pediatric cardiologist were compared to those of another pediatric cardiologist with similar expertise who scored the 15 tracks independently. There was nearly perfect agreement (14/15) for the relevant issues of murmur present/absent, murmur functional/organic, and heart disease present/absent.

Table 1.

Classification of the 15 live-recorded heart sounds according to murmur status (none, functional, organic), heart disease status (yes, no), and the diagnosis. Tracks were randomized twice to constitute pre-test and post-test versions.

Track	Murmur	Heart Disease	Diagnosis
1	none	no	Normal heart
2	none	no	Normal heart w/crackles
3	none	yes	Kawasaki's Disease*
4	functional	no	Vibratory murmur*
5	functional	no	Vibratory murmur*
6	functional	no	Venous hum*
7	organic	yes	Moderate ventricular septal defect (VSD)*
8	organic	yes	Small VSD*
9	organic	yes	Patent ductus arteriosus (PDA)*
10	organic	yes	Ebstein's anomaly*
11	organic	yes	Mitral regurgitation, congenital*
12	organic	yes	Bicuspid aortic valve w/regurgitation*
13	organic	yes	Aortic valve stenosis*
14	organic	yes	Continuous murmur (A-V malformation)*
15	organic	yes	PDA w/pulmonary hypertension*

^*Confirmed by echocardiogram.

The 15 tracks were randomized and transferred to CD-1, then re-randomized for CD-2. Each resident listened to CD-1 or CD-2 for the pre-test and to the other for the post-test. CDs were played on a PC over stereodynamic head phones (Aiwa HP-X22) in a quiet room. To control for variation in hearing acuity, the resident was allowed to adjust the volume of each recording. The test took approximately one hour. Prior to taking it, the resident received a written reminder of the characteristics of functional and organic murmurs (Table 2).

Table 2.

Definitions of murmurs as presented in writing to the resident at the time of testing.

Systolic murmurs: Occur within the S1-S2 interval. Diastolic murmurs may start at S2 (protodiastolic), as in aortic or pulmonic regurgitation, or in mid-diastole (rumbles), as in large ventricular-septal or atrial-septal defects or in mitral or tricuspid stenosis. Continuous murmurs start in systole and continue into diastole, as in patent ductus arteriosus, arterio-venous malformations, or venous hum. Functional (innocent) murmurs: Found in absence of heart defects, including 1) vibratory murmurs - short, mid-systolic murmurs with typical musical qualities; 2) venous hum, a soft, continuous murmur with diastolic accentuation; 3) Pulmonic ejection murmur, a systolic ejection murmur not preceded by an ejection click; 4) Carotid bruit, a soft, systolic ejection murmur over the carotid arteries; 5) Murmur of physiological peripheral pulmonic stenosis, a soft systolic ejection murmur widely transmitted, most commonly heard in premature infants. Organic murmurs: All regurgitant systolic murmurs, all diastolic murmurs, systolic ejection murmurs accompanied by a click, by an S2 abnormality, or by S4, and continuous murmurs (excepting venous hum). Heart disease may be present in the absence of a murmur if S2 is not split normally. Heart disease: May be indicated by any organic murmur, but may be present in the absence of murmur if S2 is abnormal, if S4 is present, or if any click is present.

For each track the resident scored the sounds and murmurs and assessed presence or absence of heart disease using a standardized scoring sheet (Table 3). The items on the scoring sheet were chosen and ordered so as to encourage the residents to learn the very basis of cardiac auscultation, viz., recognition of normal sounds and the phases of the cardiac cycle, which are often misinterpreted by the novice ausculter. The order of items on the scoring-sheet was also designed to teach the resident to follow a systematic auscultatory routine, as described previously, [10] beginning with appreciating S1 and S2 and identifying possible S3 or S4, moving on to identify possible click, and finally identifying and appreciating the significance of possible murmur. Particular sounds and murmurs that signify heart disease were highlighted as such on the scoring sheet, placing the emphasis on detection of signs of pathology rather than interpretation.

Table 3.

Scoring sheet used by the residents to record their findings and assessments for each of the 15 tracks (one sheet per track). “HD” = a reminder that the item indicates heart disease. Only the results from the ASSESSMENT column are presented in the current study.

CD number __1__ Track number __2__
FINDING		ASSESSMENT
S1	CLICK	MURMUR
□ Single	□ Absent	□ None
□ Split	□ Ejection HD	□ Functional
□ Absent	□ Mid/late systolic HD	□ Organic
S2	MURMUR	HEART DISEASE
□ Physiologically split	□ Absent	□ Absent
□ Widely split HD	□ Systolic	□ Present
□ Closely split HD	□ Diastolic HD
□ Paradoxically split HD	□ Continuous HD(except venous hum)
□ Single	□ Vibratory
□ Absent	□ Venous hum
S3
□ Absent
□ Present
S4
□ Absent
□ Present HD

Instruction

Within 48 hours of the pre-test, the instructor (pediatric cardiologist) scored the pre-test and then engaged the resident in a detailed review and discussion of each response. This instruction lasted on the average 1-2 hours. At this time the instructor also introduced the resident to a website on evaluation of pediatric cardiac sounds and murmurs, to be freely studied [http://classes.kumc.edu/som/cardiac_auscultation]. During the rotation the resident attended pediatric cardiology clinic, performing cardiac auscultation under supervision on at least 50 patients, some with heart disease, some without.

Data analysis / statistics

The comparison of the pre-test and post-test results yielded the answers to the following experimental questions: Did the residents improve at:

1. identifying a murmur when present?

2. distinguishing functional from organic murmurs?

3. diagnosing heart disease when present?

Statistics were computed using SPSS 14.0 (SPSS Corp, Chicago, Ill). Each subject's percent correct responses was calculated for each measure. Data are reported as mean ± SD of these unless otherwise noted. Because some of the outcome measures were not normally distributed, we used the non-parametric Wilcoxon signed-ranks test to assess all differences, with significance declared when two-sided p < 0.05.

Results

Twenty-six residents participated in this study. The interval between pre- and post-tests was 24 ± 9 days (median = 22 days).

Assessment of Murmurs

Figure 1 shows the accuracy of assessment of murmurs. At pre-test, recognition of the presence of murmur (Panel A) was moderate at 66 ± 17%, but increased significantly at post-test to 76 ± 16% (p = 0.007). Recognition of functional murmur (Panel B) was poor at pre-test (37 ± 33%) and improved moderately at post-test to 54 ± 27% (p = 0.048). For organic murmurs (Panel C), pre-test recognition was very good (75 ± 24%) and trended upward at post-test to 84 ± 16% (p=0.129). This apparent improvement was not statistically significant and was actually an artifact of heavy downward skewing of the pretest values, which artificially depressed the pre-test mean. The respective pre-test and post-test medians for recognition of organic murmurs were both excellent at 89% and 84% respectively. Non-normality did not confound any other comparisons. Identification of absence of murmur (Panel D) was moderate at pre-test (69 ± 35%), and it declined non-significantly at post-test to 62 ± 32% (p = 0.723).

Figure 1.

For each of 15 live-recorded heart sounds, residents judged whether a murmur was present or absent, and if present, whether it was functional or organic. Data are pre-test and post-test mean ± SEM for these assessments. P-values are for the comparison of pre-test vs post-test. Additional comparisons are reported in the text.

There was a striking difference between the recognition of functional and organic murmurs (Figure 1, Panels B & C). At pre-test organic murmurs were far more accurately identified than functional murmurs (75 ± 24% vs 37 ± 33%, p < 0.002), and although both improved at post-test, the difference persisted (84 ± 16% vs 54 ± 27%, p < 0.001).

Recognition of Heart Disease

Figure 2 (Panel A) shows that pre-test recognition of heart disease when present (sensitivity) was high, at 76 ± 19%, and improved significantly at post-test to 86 ± 13% (p = 0.016). By contrast, the recognition of absence of heart disease (specificity) (Panel B) was low at both pre-test (55 ± 19%) and post-test (59 ± 25%) (p = 0.601). Thus, even at post-test, by auscultation alone residents were far more accurate at identifying the presence of heart disease than at identifying its absence (86 ± 13% vs 59 ± 25%, p < 0.001).

Figure 2.

For each of 15 live-recorded heart sounds, residents judged whether heart disease was present or absent. Data are pre-test and post-test mean ± SEM for these assessments. P-values are for the comparison of pre-test vs post-test. Additional comparisons are reported in the text.

Discussion

At the beginning of their pediatric cardiology rotation our residents had 76% sensitivity to heart disease by auscultation, which is similar to that of general pediatricians with five to ten years of clinical experience. [9] This was quite remarkable and bodes well for their ability to recognize organic murmurs by either their quality or duration, or the presence of clicks—the “cardinal clinical signs” that are strongly associated with heart disease. [7] At post-test the residents showed significant improvements in identifying (1) the presence of murmur and (2) the presence of heart disease. At 86% their post-test sensitivity for heart disease was similar to that of pediatric cardiologists. [10] These improvements, based on a test using live recordings of children's heart sounds, are consistent with earlier results reported for simulated heart sounds, [3] and they reinforce the concept that cardiac auscultation can be taught, even during a short training period. At variance from our study, a recent report found that cardiac auscultatory skills of pediatrics residents were poor at the outset and did not improve as a result of an outpatient cardiology rotation, but did improve after using a self-directed cardiac auscultation teaching program. [5]

We believe that the improvements seen here resulted largely from the structured auscultation that was emphasized during both the testing and the rotation's actual clinical practice, and from the direct instruction that focused on weaknesses identified in the residents' pre-test performances. Practice effects from pre-test to post-test and changes in diagnostic bias may also have contributed to the improved performance. We tried to minimize practice effects by giving alternate forms of the criterion test, and no change in diagnostic bias was observed in that the overall false-positive rate was nearly identical at pre-test and post-test.

The improvement in the diagnosis of heart disease seen in this study is an important achievement because it tends to ensure that referrals will be made appropriately. Not so encouraging was the false-positive rate for diagnosis of heart disease (45% at pretest and 41% at post-test) and the misidentification of functional murmurs as organic (63% at pre-test, improving to 46% at post-test). We feel that teaching of cardiac auscultation for the primary care physician should not only stress the importance of recognizing disease but also emphasize the confident recognition of normal sounds and of functional murmurs, all of which will ensure proper referrals.

Limitations of the study

Our sample was relatively small and so may not have afforded enough statistical power to detect small but potentially important differences. Small sample size also precluded analysis of potential differences between residents in their second vs third years. Perhaps more important for methodology would be the nature of the 15 tracks of live-recorded heart sounds used as the criterion test. Nine of the 15 tracks represented loud, harsh organic murmurs, which is a substantially higher fraction than would be encountered in a general pediatric practice. This may have biased some listeners towards identifying murmurs as organic and thus may have contributed to the high false-positive rates.

Recommendations

To reduce primary care providers' excessive false-positive rates, teaching should focus on auscultation of normal heart sounds, emphasizing the absence of murmur; the instructor should work with the resident on the recognition of functional murmurs; and, throughout, should try to engender confidence in calling normal “normal” and functional “functional”. This task should not be difficult, for the great majority of functional murmurs in children are Still's murmurs. [7] These murmurs have typical vibratory (or “musical”) characteristics that render them easily recognizable and hence not difficult to teach to the novice. By contrast, the subtle nuances of cardiac auscultation are probably best left to subspecialty training.