Using a Multimedia Tool to Improve Cardiac Auscultation Knowledge and Skills

Abstract

OBJECTIVE

Today's medical school graduates have significant deficits in physical examination skills. Medical educators have been searching for methods to effectively teach and maintain these skills in students. The objective of this study was to determine if an auscultation curriculum centered on a portable multimedia CD-ROM was effective in producing and maintaining significant gains in cardiac auscultatory skills.

DESIGN

Controlled cohort study

PARTICIPANTS

All 168 third-year medical students at 1 medical school in an academic medical center.

INTERVENTIONS

Students were tested before and after exposure to 1 or more elements of the auscultation curriculum: teaching on ward/clinic rotations, CD-ROM comprehensive cases with follow-up seminars, and a CD-ROM 20-case miniseries. The primary outcome measures were student performance on a 10-item test of auscultation skill (listening and identifying heart sound characteristics) and a 30-item test of auscultation knowledge (factual questions about auscultation). A subset of students was tested for attenuation effects 9 or 12 months after the intervention.

RESULTS

Compared with the control group (1 month clinical rotation alone), students who were also exposed to the CD-ROM 20-case miniseries had significant improvements in auscultation skills scores (P < .05), but not knowledge. Additional months of clerkship, comprehensive CD-ROM cases, and follow-up seminars increased auscultation knowledge beyond the miniseries alone (P < .05), but did not further improve auscultation skills. Students' auscultation knowledge diminished one year after the intervention, but auscultation skills did not.

CONCLUSION

In addition to the standard curriculum of ward and conference teaching, portable multimedia tools may help improve quality of physical examination skills.

The current decline in physical examination skills of medical students and residents has been noted and lamented by residency directors, students, and those looking to decrease the utilization of costly medical tests.^1–3 Cardiac auscultation skills provide the clearest indicator of this trend, with today's medical students falling far short of expectations in their abilities.^4,5 The teaching of auscultation has been hampered by busy clinics, high volumes of patients,⁶ and the availability of sophisticated cardiac imaging studies that make it easier (though significantly more expensive) for a student or physician to order a test than to spend the extra time in detailed auscultation. There is mounting evidence that students are exposed to instructors who themselves have variable skills in auscultation: residents have alarmingly low identification rates of abnormal heart sounds,¹ and even experienced cardiologists have low interobserver agreement about certain cardiac auscultation findings.⁴ A 1993 survey revealed “no structured teaching of cardiac auscultation in three fourths of internal medicine programs and two thirds of cardiology (fellowship) programs.”⁵ This decline in skill is not due to a lack of evidence that accurate auscultation is a valuable tool. Recent studies have confirmed the utility of auscultation in asymptomatic valvular disease,⁷ the evaluation and management of dyspnea,⁸ and the diagnosis of congestive heart failure.⁹

Uniform delivery of an auscultation curriculum in medical school attempts to overcome the deficiencies of a curriculum taught by multiple instructors. Heart sound simulators, auscultation enhancement technology, audiotapes, and other computer-based interventions have been implemented at different medical schools.^2,5,10–15 However, where outcomes have been measured, these attempts have met with mixed results.^2,5,11,12,15 From an educational perspective, an ideal curriculum for teaching auscultation skills would be to have each student evaluate a series of live patients with abnormal heart sounds in the clinic of an expert cardiologist who would then take the time to review each case and instruct students individually. Given the numbers of students involved, the availability of patients, and the schedules of cardiologists, this curriculum is not feasible. Fortunately, advances in technology have provided an opportunity to create a simulation of this experience without the requirements of excessive faculty or patient time. Using a multimedia CD-ROM curriculum, all students can receive both the benefits of an outpatient experience and the teaching and feedback of an expert clinician.

We have developed and integrated such a multimedia CD-ROM–based educational tool into the third-year medical students' internal medicine clerkship at our institution. This program uses the principles of computer-aided instruction, presenting live patient interviews and synthesized heart sounds in an interactive learning environment to illustrate different cardiac disease states. The objective of the current study is to determine if this method of teaching auscultation improved students' auscultation knowledge and skills, and if so, to test the durability of this improvement.

METHODS

Study Participants

All third-year medical students in the July 1997 to June 1998 academic year who rotated through the required 3-month clinical rotation in internal medicine at the University of Michigan Medical School were included in this study. Historically, the division of students into this clinical rotation has been equal, so there was no expectation that any one group of students would be systematically different from another.

Auscultation Curriculum

Our curriculum for teaching auscultation contained 3 elements: ward/clinic teaching, CD-ROM–based comprehensive cases with follow-up case-review seminars, and a CD-ROM–based case miniseries. All CD-ROM cases were developed by one of the authors (RDJ).

Ward and clinic instruction consisted of residents and attending physicians teaching auscultation skills as part of a student's experience on the internal medicine rotation. This element of the curriculum can be inconsistent across study participants. However, the teachers understood that the instruction of these skills is a core expectation of the rotation.

The second element consisted of 3 CD-ROM–based comprehensive cases, each requiring about 1 hour of student interaction. Each case contains an interactive video segment of the patient interview, with some information provided by user request. Pertinent physical exam findings are then presented. For the cardiac examination, a picture of the physician listening to the patient's heart appears, and the student clicks the mouse on the picture to hear the synthesized heart sounds played through headphones (See Fig. 1). The student is asked to identify the sound, matching it to 1 of many schematic diagrams provided (phonocardiograms). He or she then listens to the correct and incorrect choices, to help reinforce subtle auscultatory differences. Pathophysiologic explanations, follow-up laboratory evaluation, radiographs, ECGs and echocardiograms are displayed to help complete the reinforcement. The 3 topics for these cases were (a) evaluation of the innocent murmur, (b) mitral valve prolapse, and (c) coronary artery disease (S4 and acute mitral regurgitation).

FIGURE 1.

CD-ROM sample screen.

An example case within the program describes a 28-year-old pregnant woman who is referred to a cardiologist for evaluation of a systolic murmur. On each screen page of the program, the student is asked to identify or interpret key physical examination findings. Incorrect answers lead to audio, video, and/or textual responses that demonstrate why the answer is incorrect. In this example case, students identify the sustained apical impulse, reduced intensity of the second heart sound, and delayed carotid upstroke of this patient with aortic stenosis (AS). Confirmation of the diagnosis is demonstrated with echocardiography. Key take-home points include 1) the auscultatory changes that occur with pregnancy, 2) appropriate use of extra maneuvers to distinguish AS from other murmurs, and 3) follow-up and management of patients with AS, including the need for periodic echocardiography and endocarditis prophylaxis. Because each student may provide different incorrect answers to different portions of each case, the instruction provided by this program is tailored to the strengths and weaknesses of the individual student.

Students were all expected to complete each of these cases, although no record was kept of the amount of time students spent on each case. After a specified time period (1 to 2 weeks), students participated in a 2-hour interactive review session of these and comparable cases, conducted by a cardiologist (RDJ). He identified the students' learning difficulties from their experience with the program, and further instructed them in cardiac examination skills.

As the third element of the curriculum, a series of 20 short cardiac physical examination cases were developed and also placed on CD-ROM. This miniseries requires approximately 2 hours to complete, and is structured in much the same way as the longer cases, with the primary emphasis on auscultatory skills. Less emphasis is placed on the history, laboratory evaluation, and imaging studies. Examples of these minicases included hypertrophic obstructive cardiomyopathy, atrial septal defect, mitral regurgitation, and the third heart sound. There were no review sessions in this component of the curriculum.

Measures of Auscultatory Skill

To assess the efficacy of the interventions, we developed a multimedia evaluation instrument, with the consultation of experts in cardiology and education to ensure content validity. The test consisted of 3 components:

1. Ten extended matching items in which students use infrared stethophones to listen to and identify common heart sounds (e.g., split S₁, S₄) and murmurs (e.g., aortic stenosis, mitral regurgitation) transmitted from a heart sound simulator.

2. Twenty multiple choice items in which students are tested on appropriate uses of the stethoscope, including choice of bell versus diaphragm, and preferred location and patient position for specific sounds.

3. Ten true-false items assessing factual knowledge about the interpretation of the sounds and murmurs (e.g., “The Valsalva maneuver intensifies the murmur of hypertrophic obstructive cardiomyopathy.”)

For the purpose of analysis, we categorized questions as testing auscultation skills (component 1) or auscultation knowledge (components 2 and 3). In order to validate this assessment instrument, a group of 5 experienced board-certified cardiologists were given this test. Their scores averaged 76% correct for the auscultation skills component, and 81% correct for the auscultation knowledge component. Both scores were significantly higher than the baseline scores of the medical students.

Research Design

All third-year students in the 1997–98 academic year participated in this study. Students were assigned to 1 of 3 cohorts on the basis of when they participated in their 3-month internal medicine rotation (see Table 1). Each cohort was evaluated using the identical auscultation knowledge and skills test before and after exposure to various elements of the curriculum. Cohort C subjects comprised the control group, exposed to 1 month of ward and clinic teaching, followed by a postintervention assessment. Students in Cohort B (the partial intervention group) were exposed to the CD-ROM case miniseries and 1 month of clinical teaching prior to their assessment. Cohort A subjects (the full intervention group) were exposed to all aspects of the auscultation curriculum before their evaluation: 3 months of ward and clinic teaching, CD-ROM comprehensive cases, and follow-up seminars. Two thirds of the students in Cohort A were also exposed to the case miniseries, but some were not because of inaccessibility to the computer program. All students in Cohorts B and C eventually received the full intervention during their clerkship, after their participation in the study was complete. Durability of knowledge and skills in Cohort A was assessed 9 or 12 months after completion of the intervention, again using the same follow-up test.

Table 1.

Research Design

Cohort	n	CD-ROM Full Cases	CD-ROM Minicases	Seminars	Wards and Clinic	Attenuation Assessed
Full intervention*
A1(7/1–9/30)	35	✓	✓†	✓	✓	✓
A2(10/1–12/31)	38	✓	✓†	✓	✓	✓
Partial intervention
B (1/1–3/31)	39		✓		✓
Control
C (4/1–6/30)	39				✓

^*Cohorts A₁and A₂combined to form Cohort A (full intervention, N = 73 students).

^†Two thirds of the students in Cohort A had access to the CD-ROM Minicase series.

Attrition in the sample (e.g., failure to complete both pre- and post-tests, withdrawal from the clinical rotation) reduced the number of participants available for final analysis from 168 to 151. Attrition rates were similar across all 3 cohorts (12.0%, 13.3%, and 9.3% for Cohorts A, B, and C, respectively).

Data Analysis

Analysis was performed by intention-to-treat. A one-way analysis of variance (ANOVA) was used for comparison of cohort effects. Change scores were calculated between baseline and postintervention and evaluated with ANOVA to assess the impact of the partial and full intervention. The Tukey Least-Significant Difference procedure was used to evaluate the statistical significance of pairwise comparisons among the 3 cohorts. Change scores were also computed for 2 subsets of the full-intervention cohort (A) between the postintervention and a final assessment at the beginning of the fourth year. This change score quantified the amount of attenuation in knowledge and auscultation skills that took place over a 9- or 12-month period following the intervention. The changes attributable to the 2 intervals were compared to each other and to zero, to determine if there was a statistically significant decline in knowledge or performance. All tests were 2-tailed (except as noted), and significant results were considered for P values of less than .05.

RESULTS

Clinical rotation performance did not differ significantly between the 3 cohorts as measured by final clerkship grades and National Board of Medical Examiners Subject Examination in medicine scores (Table 2). There were more students in Cohort A choosing internal medicine than in Cohort C (P = .05), but this difference was not significant between Cohorts A and B. Baseline performance on auscultation knowledge averaged approximately 47% correct for the group as a whole (Fig. 2), with Cohort A (full intervention) having a significantly higher mean baseline performance than did Cohort B (partial intervention) (50.3% [standard deviation (SD) = 12.5%] vs 43.2 (SD, 12.3; P < .05). At the postintervention assessment, Cohort A performed significantly better than did either Cohort B or C (A = 68.4% [SD, 11.6] vs B = 55.0% [SD, 10.3] and C = 54.7% [SD, 13.8]; P< .05) These baseline differences mandated the use of change scores, which enabled each individual to act as his or her own control. The analysis of these change scores revealed a significantly greater increase in performance on the part of the full intervention cohort (A = 18.2%; effect size [ES] = 1.6 SD units) than for either the partial intervention (B = 11.9%; ES = 1.2 SD units) or the control cohort (C = 7.8%; ES = 0.6 SD units) (P < .05). These cohort differences accounted for almost 10% of the total variance in performance.

Table 2.

Subject Demographics

		Internal Medicine Clerkship Grade†
Cohort, N	Mean NBME Subject Exam Score (SD)*	Honors, %	High Pass, %	Pass, %	Internal Medicine as Career Choice‡, %
A (n = 73)	535 (114)	12	33	55	33§
B (n = 39)	573 (112)	16	35	49	28
C (n = 39)	557 (106)	18	32	50	15§

^*ANOVA: F = 1.744, P = ns.

^†Overall: χ²(df2) = 0.990, P = ns.

^‡Overall: χ²(df2) = 3.962, P = ns; A versus B: χ²(df1) = 0.258, P = ns.

^§A versus C: χ²(df1) = 3.966, P = .05.

NBME, National Board of Medical Examiners.

FIGURE 2.

Change in performance on a 30-item auscultation knowledge test from baseline to postintervention assessment.

Performance on auscultation skills showed no differences among the cohorts at baseline (Fig. 3). At the postintervention assessment, Cohorts A and B (full and partial interventions) performed significantly better than the control cohort, but not significantly different from one another (A = 71.1% [SD, 16.5], B = 69.0% [SD, 20.6], C = 60.3% [SD, 19.4]; P < .05). The change scores also indicated a 3 to 4 times greater difference between the 2 intervention cohorts (A = 15.8% [ES = 1.0 SD units], B = 20.0% [ES = 1.0 SD units]) and the control group (C = 5.9% [ES = 0.3 SD units]) (P < .05).

FIGURE 3.

Change in performance on a 10-item auscultation skills test from baseline to postintervention assessment.

The durability of performance gains was examined in more detail through secondary analyses of students in Cohort A using a follow-up assessment of auscultation knowledge and skill administered early in the fourth year of medical school. This provided an assessment of attenuation of these gains either 9 or 12 months after the initial intervention. Analyses of variance on postintervention, follow-up assessment and change scores revealed that auscultation knowledge had decreased by 7 to 8 percentage points (P < .05) over this period, with no difference between the 9- and 12- month follow-up groups (Fig. 4). Auscultation skills, however, were preserved over this period for both subgroups (Fig. 5).

FIGURE 4.

Attenuation in performance on a 30-item auscultation knowledge test from postintervention to delayed follow-up assessment.

FIGURE 5.

Attenuation in performance on a 10-item auscultation skills test from postintervention to a delayed follow-up assessment.

DISCUSSION

In this controlled cohort study, those students who were exposed only to 1 month of an internal medicine clerkship and a self-guided CD-ROM 20-case cardiac auscultation miniseries had significantly greater improvements in auscultation skills than the control cohort of the clerkship alone. Additional months of the clerkship, with longer CD-ROM cases and follow-up cardiology seminars increased auscultation knowledge beyond the miniseries alone, but did not further improve auscultation skills. Students' auscultation knowledge diminished 1 year after the intervention, but auscultation skills did not. Our findings demonstrate the potential for a portable, self-guided multimedia instructional tool that can effectively improve a physical examination skill without reliance on a specific faculty expert.

The medical students included in this study had good baseline knowledge and skills about auscultation (approximately 50% correct on pretest scores). This moderately high initial ability compares favorably to the 20% correct reported in other settings.¹ In the Mangione study, however, students were not required to identify the nature of the sound, (e.g., late systolic murmur) but instead were provided both the sound and location of the stethoscope, then asked to identify the nature of the pathology (e.g., aortic stenosis). In addition, prior studies of auscultation instruction require the presence of an expert instructor, where ours indicates the effectiveness of an instructor-independent teaching tool.

In our study, compared to expert cardiologists, students started out approximately 35 percentage points lower on knowledge and 20 percentage points lower on skills, contributing evidence of construct validity of the assessment instrument. After the CD-ROM intervention, students demonstrated significant improvement in performance on the multimedia test of auscultation skills. While we cannot conclude that this increase in test scores corresponds to clinical outcomes, we infer clinical significance by the proximity of postintervention scores to those of expert cardiologists.

Once students learn both the auscultation knowledge and skills, we have provided evidence that their gains are durable over a 1-year period. These students had small (7%), but statistically significant decrements in knowledge, but remained 10% higher than their baseline scores 9 months and 1 year after the intervention. Despite these decreases in knowledge, there was no significant decrease in auscultation skills. While the positive gains achieved from many educational interventions return to baseline after 1 year or less, the effect of this intervention (particularly on the skill component) appears to have little attenuation over time.

In comparison to other teaching methods, computer-aided instruction has been shown to be as effective as more traditional methods, but more efficient at teaching information. Students appear to enjoy learning and are more motivated to learn using computer-aided instruction.^16–20 They can use these tools at their own convenience, without time constraints on their learning, allowing for repetition and review. By progressing at their own pace, students can then adapt the instruction to their individual learning preference.²¹

While we believe that this curriculum has been quite successful, there are limitations to our conclusions. First, the students in Cohort A had higher baseline knowledge scores than those in Cohort B. However, we could find no systematic differences between these 2 groups of students (by examination, clerkship grades, or career choice) that would explain our findings. Second, we have only studied students at 1 institution for 1 academic year. Further analysis at other sites will help support the generalizability of our conclusions. Third, this study was not designed to determine whether a similar amount of time spent reading a textbook, attending a lecture, or listening to patients' hearts would be equally effective. Fourth, it is unclear whether a student's performance on a multimedia test would correlate with the ability to accurately detect heart sounds in a patient at the bedside. In addition, it is difficult to attribute the lack of attenuation at 9 and 12 months to the effect of the CD-ROM alone using this study design. While the curriculum caused improvements in knowledge and skill, the durability in skill and small deterioration in knowledge occurred in the context of much additional practice, experience, and possibly additional teaching during subsequent rotations.

As hospitals and clinics become increasingly busy, and as patients spend fewer days in the hospital, faculty have less time for teaching. Younger attending physicians are spending less time with bedside teaching and more time in conference rooms.^22–24 Resident-teachers have been found to have a limited ability to identify heart sounds, so would be dubious teachers of auscultation skills.^5,25 This study has shown promise for the use of computer-based applications, in conjunction with clinical teaching, in the future of medical education.