Mastering Electrocardiogram Interpretation Skills Through a Perceptual and Adaptive Learning Module

Abstract

Although accurate interpretation of the standard 12‐lead electrocardiogram (ECG) is fundamental to diagnosing heart disease, several prior studies report low accuracy rates among medical students, residents, and practicing physicians. The objective of this study was to determine if an online ECG Perceptual and Adaptive Learning Module (ECG PALM) is an efficient instrument to teach ECG interpretation.

The ECG PALM consists of 415 unique ECG tracings with associated pretest, posttest, and delayed tests, each using 30 additional ECGs to gauge the effectiveness and durability of training. Between 2013 and 2015, a total of 113 third‐year and 156 fourth‐year medical students and 34 first‐year, 41 second‐year, and 37 third‐year emergency medicine residents completed the PALM and associated tests. We measured two mastery criteria: accuracy, the percentage of correct interpretations, and fluency, the percentage of images interpreted accurately within 15 seconds.

The ECG PALM produced statistically significant improvements (0.0001 < p < 0.0045) in student and resident performance for both accuracy (effect size = 0.9 to 3.2) and fluency (effect size = 2.5 to 3.1) following training ranging from 46 ± 24 minutes (R3s) to 88 ± 32 minutes (third‐year medical students). Medical students and residents performed significantly better on a test the year following training (delayed test) than those without prior ECG PALM training (pretest). The fluency of R3 residents in classifying the 15 diagnostic categories was less than 60% for nine of the 15 diagnoses and greater than 80% for only one. Following PALM training, fluency was higher than 80% for seven of the 15 categories and less than 60% for only two categories. Accuracy in recognizing ST‐elevation myocardial infarctions (STEMIs) was high both before and after PALM training for R3s, but fluency was only 64% for anterior STEMIs on the pretest, increasing to 93% following PALM training.

These observations suggest that the ECG PALM is an effective and durable supplemental tool for developing mastery in interpreting common ECG abnormalities.

Competence in interpreting electrocardiograms (ECGs) is critical to effective diagnosis and treatment decisions of cardiac abnormalities. Indeed, ECGs are the most common tool for diagnosing such abnormalities. ¹ Of particular importance to emergency medicine physicians is the ability to discriminate acute cardiac abnormalities. ECG interpretation is dependent on pattern recognition, as are many facets of medicine. Expertise in clinical pattern recognition typically arises from years of experience and has been difficult to teach using conventional instructional methods. ² , ³ , ⁴

The ability to quickly and accurately interpret patterns such as those present in ECG tracings is an example of perceptual learning. ⁵ , ⁶ , ⁷ Perceptual learning is defined as experience‐induced improvements in the pickup of information. ⁵ In this and many other areas of advanced expertise, perceptual learning involves domain‐specific improvements in the way the brain extracts information that tend to optimize task performance. These changes include a variety of information extraction improvements, broadly separable into discovery and fluency effects. ⁶ Discovery effects involve finding the information relevant to important classifications, extracting it selectively, and distinguishing relevant features from irrelevant variation. Fluency effects involve extracting and encoding relevant information more easily. These include, most obviously, speed but also more parallel processing and automaticity (lower cognitive load). ⁶ The emerging literature on perceptual learning in complex domains suggests that: 1) changes in the way information is extracted comprise a larger component of expertise than often realized, ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ 2) perceptual learning works synergistically with other cognitive and learning processes, ⁷ and 3) this kind of learning is hard to engage effectively with conventional instructional methods. ⁹

Research has demonstrated an association between longer response time (RT) and lower accuracy in interpreting visual presentations such as mammograms ¹¹ and skin cancer ¹² as well as in diagnosing a series of written cases. ¹³ Verbal or procedural instruction, or consideration of a few examples, ais re not well suited to advance this type of learning, which may often be implicit (unconscious) and advance through classification episodes involving relevant kinds of variation in the target domain. ⁶ , ¹⁴ , ¹⁵ Attaining expertise in classifying new cases accurately and doing so fluently typically requires many years of experience to see enough cases for such distinctions to be developed.

Currently, undergraduate medical curricula include few approaches optimized to enhance pattern recognition in ECG interpretation. ² , ¹⁶ In graduate medical education, residents increase their ability to interpret ECGs via experiential learning, ¹⁷ but this learning is based on viewing cases that present in a random fashion, are heavily skewed toward the most common presentations, and are independent of the learner’s areas of weakness. Despite development of greater proficiency during this period, there is evidence that residents in emergency medicine, ¹⁷ , ¹⁸ , ¹⁹ internal medicine (IM), ⁴ , ¹⁸ cardiology, ²⁰ and family practice ²¹ , ²² exhibit higher error rates than anticipated given their years of study and than needed given their levels of responsibility. The low level of proficiency and self‐confidence of early‐stage IM residents at one medical institution has led authors to explicitly “emphasize the need for improved and more effective training in ECG interpretation for physicians starting residency.” ⁴

We have developed and implemented online perceptual and adaptive learning modules, or PALMs, in our undergraduate and graduate medical curricula. These were targeted at increasing the rate of development of expertise and evaluating competencies in several pattern recognition‐dependent domains. We found that these PALMs produced large and durable improvements in interpretation of clinically relevant images and video clips. ²³ , ²⁴ , ²⁵ When surveyed, medical students also responded very favorably to these interventions, indicating that PALMs increased their sense of confidence and their interpretation skills and strongly endorsed the idea that they should be included more broadly in the curriculum. ²³ , ²⁴

A recent systematic review of educational software aimed at teaching ECG interpretation concludes that more and better studies are needed to evaluate the efficacy of such approaches. ²⁶ The purpose of this study was to determine if a PALM developed for 12‐lead ECG interpretation is an efficient and effective method of teaching ECG interpretation, based on the principals of perceptual learning, and leads to improved pattern recognition and prolonged retention of this skill.

METHODS

PALM General Principles

A PALM presents a series of single‐answer multiple‐choice classification trials. To the learner, the process appears similar to a flashcard program; however, there are significant features built into PALMs that optimize the development of pattern recognition rather than recall of specific examples. These features can be summarized as follows:

• Stimuli are grouped into categories according to their diagnostic interpretations, with enough instances of each category that a given image will have a low probability of being seen more than once in the course of completing the learning module.

• PALMs adapt to the learner via an ARTS (adaptive RT‐based sequencing) system, ²⁷ which uses learner performance to adaptively space and sequence learning categories. ⁷ The ARTS system adjusts the priority of seeing an exemplar of a given category higher for a longer RT on the category’s previous presentation and for more intervening trials since that presentation; the priority is also increased more for incorrect than correct answers. Spacing based on the combination of accuracy and RTs in ARTS has been shown to outperform other spacing schemes in both factual learning ²⁸ and perceptual learning. ²⁷

• RT is included as a criterion in assessing the degree of mastery, or “learning strength,” ²⁷ of each category with category mastery being reached once a preset number of consecutive exemplars of the category are identified both accurately and within a predetermined target RT. Categories are retired once they are mastered, and the PALM is complete once all categories are retired.

ECG PALM and Test Development

This ECG PALM consists of a series of interactive learning trials in which exemplars of 15 nonarrhythmia diagnostic categories were presented, and learners made a classification response on each trial. We (authors JN, CS) selected 505 twelve‐lead ECG tracings from departmental teaching files for use in the ECG PALM (415 images) and in the pretest, posttest, and delayed tests (30 additional images per test). The interpretation of each of the ECG images was confirmed by two board‐certified emergency medicine physicians (authors JN and CS).

Fifteen common diagnostic categories were selected: anterior and anterolateral ST‐elevation myocardial infarction (STEMI), inferior inferoposterior STEMI, old/indeterminate anterior MI, old/indeterminate inferior MI, left atrial enlargement, right atrial enlargement, left ventricular hypertrophy, right ventricular hypertrophy (RVH), left bundle branch block, right bundle branch block, left axis deviation, right axis deviation (RAD), sinus bradycardia, sinus tachycardia, and normal sinus rhythm. These ECG diagnoses were selected because they best fit the construct of perceptual learning, which in simplest terms is the development of pattern recognition. We attempted to select images that primarily exemplified a single interpretation; in some cases (e.g., RVH) secondary findings (e.g., RAD) commonly accompany the condition and are thus part of the pattern that the PALM users should learn to recognize for that classification. An example of a PALM trial along with feedback is illustrated in Figure 1, in this case, for a correct answer choice. For a correct answer only, the RT is also displayed.

Figure 1.

Examples of PALM trials and feedback for incorrect and correct answers. An incorrect answer choice (A) is followed by feedback showing the incorrect choice highlighted in red and the correct choice highlighted in green (B) along with a verbal description of the criteria for the correct interpretation and arrows and parallel lines to draw attention to features and magnitudes of differences relevant to the interpretation of the tracing (and arrowheads indicating variable features). In the case of a correct answer (C), the response time is also indicated and only the correct answer is highlighted. See Results for a more detailed discussion of this figure. PALM = perceptual and adaptive learning module.

In addition to the ECG PALM, we developed three different assessment versions, each consisting of two different, unique exemplars of each category (30 questions per assessment) to be used as pretests, posttests, and delayed tests in assessing the effectiveness of the ECG PALM in boosting and maintaining ECG interpretation skills. The three versions of the test were distributed evenly among trainees for the pretest, posttest, and delayed test, whereas the three test versions were rotated for an individual trainee on successive assessments so that the learner never saw the same test version on more than one assessment. Distributing the three assessments in this manner controlled for the possibility that observed outcomes were influenced by different intrinsic levels of difficulty between the pretest, posttest, and delayed test problems.

Study Design

The study used a repeated measures design, with participant accuracy and fluency on pretests, posttests, and delayed tests (relative to completion of the ECG PALM) being the dependent variables measured. Fluency, determined by the number of trials interpreted both accurately and within the target RT, disregarded trials on which responses were inaccurate, as crediting fast responses as good performance when the response is incorrect is misleading and fails to enhance mastery. Also, note that fluency is a subset of accuracy determined by a target RT cutoff, but the value of its actual calculation does not include RT; thus, accurate responses made with lower RTs are more fluent. Participants were allowed to complete the PALM on their own schedules (but within a predetermined range of dates) wherever they had Internet access. The PALM training could be broken up into as many sessions as the participant chose and upon reengaging would start where the participant left off. Although PALM training and the associated assessments were a required component of their training, the pretests, posttests, and delayed tests served as formative assessments that did not count toward any aspect of trainee evaluation or advancement.

The user was allowed 30 seconds in which to make an answer choice before a trial timed out and the feedback was displayed (a timed‐out trial was scored as incorrect). The target RT for an accurate answer to count towards category retirement was set to 15 seconds. In arriving at the target RT value, we doubled the RTs of three clinicians involved in teaching ECG interpretation to clerkship students and emergency medicine residents on this PALM after subtracting 2 seconds for the process of finding and moving the cursor to the correct answer choice. This approach was in accord with that used in previous PALM‐based studies. ²³ , ²⁴ , ²⁵ The 30‐second limit, which was estimated to allow enough time to analyze the ECG, provided the participant recalled the features on which to base the interpretation. The calculated target RT along with the 30‐second time limit were endorsed by the three clinicians involved in teaching ECG interpretation to clerkship students and emergency medicine residents. There is no implication that such a target RT identifies an actual “boundary” between pattern recognition and explicit conscious analysis as these processes are most likely highly overlapping, “recognition” gradually developing with experience in the process of image “analysis.”

We defined an adequate level of “mastery” for these groups of trainees as accurate identification of three consecutive exemplars of a particular category within the target RT. Successive presentations of exemplars from the same diagnostic category were spaced by a minimum of three intervening trials. Once the number of active categories remaining was too small to allow for three intervening trials, exemplars from the previously retired categories were used as “filler” trials to provide spacing; responses to these “filler” trials were not included in determining category sequencing or retirement state.

We evaluated two types of performance measures: “accuracy” and “fluency.” A correct answer occurring at any time during image presentation (30 seconds) was given a point for accuracy, whereas only those correct answers occurring within the predetermined target RT (15 seconds) were given a point for fluency. In this way, fluency is similar to its use in characterizing the level of mastery of a spoken language, with “fluent” denoting both accurate and automatic. Fluency is more indicative of advanced pattern recognition while accuracy includes correct responses based on discovery processes and perhaps more explicit feature analysis.

Participants

The online ECG PALM, along with the pretest and posttest, was required of third‐year medical students enrolled in the inpatient IM clerkship between May 16, 2013, and May 25, 2014 (N = 113 total and does not include students in two groups of IM rotations who were asked to complete a different version of the ECG PALM). At the beginning of their fourth year intensive care unit (ICU) rotations (August 2, 2013, to April 10, 2015), 110 of these students completed a delayed test. The period between completion of the posttest in the third‐year IM rotation and the delayed test at the beginning of the student’s fourth‐year ICU rotation was a mean (±SD) of 343 (±151) days and was used to examine long‐term retention of PALM‐induced ECG interpretation mastery. The ECG PALM and associated pretests and posttests were also completed between July 19, 2013, and April 11, 2014, by fourth‐year students, during ICU rotations, who had not previously completed this PALM in their third‐year IM rotation (n = 143).

Finally, the ECG PALM was completed between September 10, 2013, and September 18, 2015, by first‐year (R1; n = 24), second‐year (R2; n = 44), and third‐year (R3; n = 37) emergency medicine residents training at an academic, university‐affiliated medical center.

The numbers of participants for each training year include those who had completed this PALM during 2 consecutive years of training; however, the analyses distinguish between first‐time and repeat‐exposure participants. This study was approved by the UCLA Institutional Review Board (#11‐002361).

Data Analysis

Outcomes analyzed were accuracy and fluency. Differences between pretests, posttests, and delayed tests for each class were assessed by correlated t‐tests after confirming the existence of significant differences using one‐way ANOVA tests. Between‐group differences were assessed via an independent t‐test. Two‐tailed p‐values were calculated with statistical significance indicated by α < 0.05. (http://vassarstats.net/). Effect size was calculated as Cohen’s d,

$${\text{Cohen}}^{\prime }\mathrm{s}d=M_1-M_2/{\sigma }_{\mathit{pooled}},$$

where

$${\sigma }_{\text{pooled}}=\surd \left[\left(\sigma 12+\sigma 22\right)/2\right],$$

where σ is the SD of the group and M is the mean of the group.

We used Cohen’s qualitative classifications in describing effect sizes (<0.2 small, 0.2‐0.8 medium, >0.8 large). Linear regression analysis was used (http://vassarstats.net/) to assess the rate of decay and associated 95% confidence intervals (CIs) of PALM‐induced learning.

RESULTS

The pretest and posttest results for trainees completing the ECG PALM for the first time are shown in Figure 2, with accuracy (Fig. 2A) and fluency (Fig. 2B) data. The mean time taken to complete the ECG PALM ranged from 46 ± 24 minutes (EM R3 residents) to 88 ± 32 minutes (third‐year students). Based on the data in Figure 2, three main observations can be made: 1) The pretest values for both accuracy and fluency increase with increasing stage of training, suggesting face validity of this type of test as a measure of ECG interpretation skill; 2) there is little difference between the pretest performances of third‐year medical students, who completed the PALM at the beginning of their IM clerkship, and those of fourth‐year medical students who have completed all of their clerkships, suggesting that the IM clerkship itself added little value to the long‐term maintenance of ECG interpretation skills; and 3) there are statistically significant (<0.0001 = p ≤ 0.0045) and large (0.9 ≤ d ≤ 3.2 for accuracy; 2.5 ≤ d ≤ 3.1 for fluency) improvements in performance at all levels of training following PALM training.

Figure 2.

Effect of the ECG PALM on ECG interpretation skill. The percent accuracy (upper subfigure) and fluency (lower subfigure) are shown as a function of educational level for ECG interpretation before (gray bars) and after (black bars) completing the ECG PALM for the first time. The numbers of participants are shown below the training level on the abscissa. For all groups and measurements, the posttest performance was statistically greater than the pretest performance with p < 0.0001 for all pre‐/posttest comparisons except for R3 accuracy, for which p = 0.0045. ECG = electrocardiogram; PALM = perceptual and adaptive learning module.

While enrolled in the 3‐week ICU rotation (July 19, 2013, to April 10, 2015) fourth‐year medical students were asked to complete the pretest, ECG PALM, and posttest during the first 2 weeks of the rotation and to complete a delayed test on the final day of the (3‐week) rotation. For students who had previously completed this PALM as third‐year students (May 16, 2013–May 25, 2014), the pretest during their ICU rotation was equivalent to a delayed test (since pretest, posttest, and delayed test administration each used all three versions of the test). Figure 3 compares the pretest accuracy and fluency for the fourth‐year students and R1, R2, and R3 trainees who had not completed the ECG PALM the previous year with the pretest, given at approximately the same point in their training (but during the subsequent year), for those who had completed the PALM the previous year (thus serving as a “delayed test” for this latter group of trainees). Because third‐ and fourth‐year students rotated in different groups, staggered through the various clerkships and ICU rotations, the delay between their previous PALM and the pretest as fourth‐year students, 343 ± 151 days, had a broad range of delays. The R1 trainees who completed the PALM in their previous year had all done so as fourth‐year students, and their delay, 268 ± 104 days, had a range, determined by which ICU rotation they had participated in. The R2 and R3 residents completed the PALM and associated tests between mid‐July and mid‐September of the training year; the range of delay between their initial PALM training as residents and delayed testing and training in subsequent years was 399 ± 14 days for EM R2s and 366 ± 50 days for EM R3s. With the exception of the accuracy performance of R3 residents (for which p = 0.07), both accuracy and fluency scores for trainees who had completed the PALM approximately 1 year earlier was statistically significantly higher than those for trainees at the same educational level but with no previous ECG PALM training (see p‐values along the abscissa). Effect sizes for the effect of ECG PALM completion in the previous year for the fourth‐year students, R1, R2, and R3 groups for accuracy were d = 0.7, 1.7, 0.5, and 0.7, respectively, and for fluency were d = 1.3, 1.9, 1.8, and 0.8 respectively. Note that, in addition to being higher than for those at an equivalent level of education but without the previous ECG PALM training, the delayed performance of fourth‐year students and residents with the previous PALM training was also comparable to (accuracy) or higher than (fluency) that of trainees at the next higher level of education without previous ECG PALM training.

Figure 3.

Effect of previous versus no previous ECG PALM training on ECG pretest performance. Gray bars represent performance of trainees having no previous ECG PALM experience on an ECG PALM pretest given at the beginning of the training year, and black bars represent performance on the same test for trainees completing the ECG PALM the previous year (i.e., the pretest functions as a delayed test for this latter group). The number of trainees in each group is shown on the abscissa under the training level along with the p‐values for statistical significance of performance differences between those with and those without ECG PALM experience the previous year. For accuracy, effect sizes were d = 0.7, 1.7, 0.5, and 0.7, and for fluency, they were d = 1.3, 1.9, 1.8, and 0.8 for MS4, R1, R2, and R2 trainees, respectively. ECG = electrocardiogram; PALM = perceptual and adaptive learning module.

Because of the large number of students completing the ECG PALM during both their third and their fourth years of training, we were also able to assess a rate of decay over a 635‐day period for the PALM‐induced learning. Figure 4 plots, for each student, the difference between their delayed posttest and their initial pretest scores, for both accuracy and fluency, as a function of the number of days since completing the posttest. The line in each data set is fit by a linear regression, the slope of which suggests that the ECG PALM‐enhanced fluency would decay back to the pretest value at about 13.9% per year, whereas the accuracy would decay back to its pretest value at about 13.1% per year. An additional benefit of using this set of participants is that they had much less actual experience interpreting ECG’s during the period between their initial ECG PALM training in their third year and the delayed testing in their fourth year of training; so the decay is less influence by additional experience in ECG interpretation.

Figure 4.

Decay of ECG PALM–trained accuracy and fluency. Each circle represents the difference between the performance of an individual medical student on an assessment following ECG PALM training relative to that student’s performance on the pretest during their year 3 internal medicine rotation. The abscissa indicates the number of days since the student completed the delayed test. The initial values at 0 days is the difference between the pretest and posttest values for all of the third‐year students who completed the delayed test in their fourth year, whereas the points at the later times, between 76 and 635 days, represent the differences in performance between the delayed test and the pretest for the same group of students. The lines represent the linear regression best fit to the data. For accuracy, slope = −0.036% (95% CI = –0.046 to –0.027%) and R2 = 0.22 and for fluency, slope = −0.038% (95% CI = –0.048 to –0.029 and R2 = 0.23. ECG = electrocardiogram; PALM = perceptual and adaptive learning module.

We examined the performance of the 25 students enrolled in the first IM rotation of the 2013 to 2014 academic year to see whether their performance immediately following the pretest was increased to a level comparable to that seen on the posttest. This IM rotation group was chosen to avoid the influence of ECG training that might have occurred during other year 3 clerkships. We compared the average performance on the 30 pretest ECGs and 30 posttest ECGs with their performance on the first and last 30 ECG presentations in the PALM. We found that performance for this group on the pretest (accuracy = 45%, fluency = 26%) was very similar to that for the first 30 trials of the PALM (accuracy = 53%, fluency = 34%), whereas performance on the final 30 trials of the PALM were markedly higher than for the first 30 PALM trials (accuracy = 81%, fluency = 69%) while being comparable to those on the posttest (accuracy = 80%, fluency = 63%). Thus the improved performance in the posttest was a result of learning processes occurring during PALM training and not simply via retrieval during the pretest of previously learned information.

Although our focus was on evaluating the effectiveness of this PALM in improving interpretation of ECG morphology at multiple levels of medical education, it is of particular interest also to look at performance in interpreting ECGs according to the ECG diagnosis, as well as at the relative effectiveness of the PALM in improving these. For example, it is possible that the R3 residents already perform well on the most critical ECG interpretations, such as the STEMIs, and that the ECG PALM mainly improved recognition of less critical ones.

We focus here particularly on fluency (as opposed to overall accuracy) as an indicator of category recognition. In addition, of course, the demands of making rapid diagnoses, endemic to the emergency department environment, are more likely to affect fluency of less experienced residents than more experienced physicians.

Figure 5 shows the fluency and accuracy of R3 residents in classifying the ECG tracings prior to (Figure 5A) and immediately after (Figure 5B) completing the ECG PALM for the first time. For comparison, the fluency and accuracy are shown (Figure 5C) for the beginning R1s who had completed ECG PALM training during the previous year. Note that, interpretation of every diagnostic category improved; the mean increase in fluency was 32% and was greatest for those diagnoses with the poorest initial performances. Fluent recognition of nine of the 15 diagnoses was less than 60% prior to PALM training (dark gray) with only one diagnosis higher than 80% (light gray). The pretest accuracy of trainees was much better, however, with seven of the 15 diagnoses being identified correctly at higher than 80% and only two less than 60%. PALM training produced marked improvement in both fluency and accuracy with eight of the 15 diagnoses being recognized with more than 80% fluency and only two less than 60% and with accurate identification of 13 diagnoses higher than 80% and none less than 60%. Of particular interest is performance on the critical finding of ST‐elevation. Whereas inferior/inferoposterior STEMIs were quickly recognized by R3 trainees prior to PALM training, the anterior/anterolateral STEMI’s were not, fluency being at 64%. By contrast, accuracy in identifying both STEMIs was high, suggesting that the trainees could “figure out” the correct diagnoses but just needed longer for analyzing the anterior/anterolateral ones. Fluency was improved for both classes of STEMIs by PALM training; however, rising to 89% for the anterior/anterolateral STEMIs and to 100% for the inferior/inferoposterior ones.

Figure 5.

Influence of the ECG PALM on the performance of EM residents on each ECG diagnostic category. Accuracy and fluency of R3 residents prior to (A) and immediately following (B) completion of the PALM is shown for each category. To facilitate comparisons, performance ranges are indicated by cell shading: light gray indicates more than 80%, dark gray indicates less than 60%, and white indicates 60% to 80%. The durability of PALM training on accuracy and fluency in interpreting ECG morphology as a function of the category can be deduced from the performance of R1 residents who had completed the ECG PALM as MS4s the year previously (C). ECG = electrocardiogram; PALM = perceptual and adaptive learning module.

Although determining an “adequate” level of fluency in identifying STEMIs depends on many factors, the values attained after the PALM intervention were also notably higher than those obtained via the various computer algorithms currently in use for identifying STEMI patients in the prehospital environment. ²⁹ Fluent interpretation of left and right ventricular hypertrophy, important conditions to recognize, was only 36 and 11%, respectively, prior to PALM training but increased to 46 and 61%, respectively, after training. Although this posttraining level represents a substantial improvement, recognition is still quite low; so it seems likely that further training would benefit performance in recognizing these ECG patterns.

To examine how durable retention was for each of the diagnoses, we carried out the same analysis (Table 1C) of fluency and accuracy according to diagnosis for those R1 trainees who had completed the ECG PALM as fourth‐year students during the previous year. Interestingly, fluency was either higher than 80% or less than 60% on a given diagnosis, with six categories being recognized fluently at higher than 80%. For accuracy, nine categories were higher than 80% with only three being less than 60%. The good news is that retention of training for both the anterior/anterolateral and the inferior/inferoposterior STEMIs was apparent for both fluency and accuracy.

DISCUSSION

As the medical education paradigm pivots from general competencies toward entrustable professional activities (EPAs), the ability to assess expertise in interpreting clinical tests and key features of patient scripts and presentation, for example, will rise in importance. Of the 13 core EPA’s identified by the American Association of Medical Colleges, ³⁰ at least two would appear to require skills in ECG interpretation, notably EPA 3, recommend and interpret common diagnostic and screening tests, and EPA 10, recognize a patient requiring urgent or emergent care and initiate evaluation and management.

This study evaluated a new tool based on perceptional learning principles that would improve interpretation skills among learners at various stages of training while substantially decreasing the time needed to achieve higher skill levels. The study achieved its major objective: participants who completed the PALM testing and training modules in addition to conventional training showed significant improvement in performance with relatively modest investments of learners’ time (significantly less than 2 hours for the average user to complete the pretest, training, and posttest modules) when compared to participants trained using conventional methods.

The relatively brief PALM intervention produced large effect sizes for both accuracy and fluency in interpreting ECGs that appeared to decay very gradually over time, even in the absence of significant clinical experience. It would be hard to argue that the additional benefit of the PALM training was simply a result of the extra time (approximately ¾ to 1½ hours) spent examining ECGs since, for example, even after an approximately two‐thirds to 1‐year delay following PALM training, the intervention group performed at or slightly higher than the same level in their accuracy of ECG interpretation as non–PALM‐trained medical students and residents who were 1 year more advanced or, in their fluency, well above the non–PALM‐trained comparison group that was 1 year further along in training (cf. results of Figure 3).

The work described here, and research in other medical and nonmedical learning domains, ¹⁴ , ²³ , ²⁴ , ²⁵ , ³¹ , ³² demonstrate that learning can be accelerated if learners are exposed to numerous, varied exemplars within each diagnostic category, over a fairly short time, with contrasting categories being interleaved rather than presented in blocks and with appropriate feedback. With adaptive methods incorporating fluency as well as accuracy in tracking performance and retiring categories, perceptual learning can be made highly efficient, can focus effort where it is needed for each learner, and can implement meaningful learning criteria. This approach to learning can simply not take place in the typical clinical setting in any efficient manner.

Our focus on the perceptual learning components of expertise is not meant to suggest that understanding the physiology and pathology that underlie various ECG presentations, or the rules governing their interpretation, are unimportant or in any way dispensable. Rather, once the learner has acquired these declarative and procedural aspects of learning, there should be a systematic focus on targeted and systematized experiential learning, adapted to the individual learner and organized to maximize the transition from slower, more deliberate analysis.

Although our study does not confirm improved clinical practice or outcomes with respect to ECG interpretation in clinical settings, we have shown that it is possible to lay the foundation for speedy, accurate interpretation of common ECG abnormalities manifesting as changes in their morphology or sinus rate and that this effect is sustained over a prolonged time period. These results suggest that it would be highly worthwhile to develop and implement additional ECG PALMs for such diagnoses as those subsumed within arrhythmias and metabolic disorders. More generally, they suggest that PALMs can contribute importantly to the acquisition of “entrustability” for any clinical task dependent on accurate pattern recognition in addition to improving performance on high‐stakes medical examinations. ³³

Based on the results of our studies, for straightforward ECG presentations, the perceptual learning challenges lie well within reach of medical students and generalists. We anticipate that the PALM approach will allow for the development of similar skill levels by other providers, including midlevel clinicians, nurses, paramedics, and even the technicians who perform ECGs and other diagnostic tests. Ensuring an increased level of pattern recognition among frontline primary care providers might result in dramatically fewer specialist referrals, a major driver of health care costs.

LIMITATIONS

One limitation of this study, as discussed under Methods, is that certain critically important diagnoses were not included, specifically common arrhythmias. Given the positive outcomes observed in this study, we have begun to test a second ECG PALM based on arrhythmias. An area of possible concern relates to the online, unmonitored format that might allow participants to simply “look up” the correct diagnoses; however, the limited RT for each problem (30 seconds), and even more, the short (15 seconds) target RT reflecting fluency, would have made it difficult for a user to try to look up information in texts or online resources and apply it to the particular ECG traces being displayed. In addition, the formative nature of the exercise would have reduced motivation for doing so. Another limitation of the study is that there is no “sham” control group, the intervention being supplemental to the normal curriculum that all of the trainees receive. Finally, this study is limited by the fact that medical students from only one institution and residents in emergency medicine at only one institution were included. Thus, the effects of this ECG PALM in enhancing learning may reflect particular deficits in how interpretation of these types of ECGs is normally taught at these particular institutions. However, as discussed in the introduction, a large number of reports indicate that the ability of residents to interpret ECGs in several specialties and at several institutions is inadequate, suggesting that this problem is likely to be broad‐based and that a perceptual and adaptive type of training, utilizing correct‐answer RT as an indicator of learning strength might be widely beneficial.

CONCLUSION

Training electrocardiogram interpretation using perceptual learning approaches and a response time–based adaptive learning algorithm can produce large and durable increases in skills for both undergraduate medical students and postgraduate residents. These approaches might also be useful for maintaining electrocardiogram interpretation skills among practicing clinicians.

We are indebted to Tim Burke and Zhen Gu for PALM application development, to Kathryn Devera for technical help, and to Khanh‐Phuong Thai for many insightful discussions.

AEM Education and Training 2021;5:1–11