Personalized, assessment‐based, and tiered medical education curriculum integrating treatment guidelines for atrial fibrillation

Abstract

Background

This continuing medical education (CME) curriculum utilizes the Learner Assessment Platform (LAP), providing learners with personalized educational pathways related to atrial fibrillation treatment.

Hypothesis

There are improvements in knowledge among physician learners after CME, especially among LAP learners.

Methods

In this LAP‐based curriculum, an evaluation of learner deficits on designated learning objectives was conducted in tier 1 and used to direct learners to individualized tier 2 activities. Performance was assessed across learner tracks from baseline to learners’ final intervention. Retention data were measured by the postcurriculum assessment, completed 8 weeks after the learners last intervention. Additionally, each activity included a unique matched set of pretest and post‐test questions assessing the 4 learner domains: knowledge, competence, confidence, and practice patterns.

Results

Significant learner improvement was measured across the curriculum over all 4 learner‐domains: 48% (P < 0.0005), 78% (P < 0.0005), 21% (P < 0.0005), and 20% (P < 0.0005) improvements for knowledge, competence, confidence, and practice, respectively. Significant gains in participant performance scores (28% increase, P < 0.0005) by the final activity was observed. Learners who participated in the LAP (N = 989) demonstrated greater improvement in performance from baseline compared to non‐LAP learners (41% increase for LAP vs 23% and 26% increase for non‐LAP learners who completed 1 (N = 1899) or ≥2 (N = 533) activities, respectively, P = 0.003).

Conclusions

The participant population (N = 3421) achieved statistically significant improvement across the curriculum, with LAP learners showing greater performance gains compared to non‐LAP learners. These findings support the value of the LAP methodology in providing a cumulative and individualized CME experience.

1. INTRODUCTION

Atrial fibrillation (AF), a common cause of arrhythmia, carries a high risk of stroke and other heart‐related complications.1 Current treatment guidelines describe the use of thromboprophylaxis, rate control, and antiarrhythmic drug therapy for managing patients with AF.1, 2 The treatment guidelines address the selection of appropriate treatment options in the presence of comorbidities in the AF population, and describe safe and effective strategies for switching antiarrhythmic drugs in AF patients.2

AF prevalence in the US is near 5.2 million and projected to increase to 12.1 million cases in 2030.3 The diversity of healthcare providers faced with managing this increasing population is varied. These diverse healthcare providers have differences in training, specialty, patient populations, and knowledge gaps, suggesting a need for AF‐related continuing medical education (CME) that is targeted and individualized for each learner. The Centers for Disease Control and Prevention (CDC) advocates CME programs to improve timely diagnosis of AF and implementation of guideline recommendations to reduce morbidity and death.1

Research into CME effectiveness indicates that multiple educational techniques are more successful than traditional didactic sessions alone at improving physician knowledge and patient health outcomes.4, 5 In this study, a multitiered CME curriculum, combining live didactic and online case‐based educational activities, was developed. An individualized educational experience addressing identified learning gaps was provided. The program content consisted of selection and initiation of antiarrhythmic therapy for AF patients with different comorbidities and clinical presentations, and application of newly released guideline recommendations for antiarrhythmic and antithrombotic therapy for the management of AF patients.

2. METHODS

2.1. Atrial fibrillation continuing medical education curriculum

A multiplatform CME curricula was developed, consisting of 7 activities overall, which focused on guidelines and treatment approaches for AF. The CME curricula was accredited by the Accreditation Council for Continuing Medical Education (ACCME). The curriculum was available between October 2011 and February 2012. The target audience was primary care physicians, office‐based cardiologists, internists with a demonstrated focus in cardiology, and heart rhythm specialists involved in the management of patients with AF. The CME program was titled Advances in the Treatment of Atrial Fibrillation: Integrating Current Guidelines and Novel Therapies to Optimize Patient Outcomes. Specific learning objectives were the following:

1. Describe the role and significance of rhythm control in AF patients with a history of decompensated heart failure or in congestive heart failure patients who are symptomatic

2. Implement strategies for safe and effective switching of antiarrhythmic drugs (AADs)

3. Recognize various contraindications to the specific use of AADs to reduce risk of adverse effects and complications

4. Develop AAD treatment plans that take into account safety, efficacy, and patient‐specific attributes in the selection, initiation, and ongoing management of AAD therapy

5. Incorporate new guidelines for treatment utilizing specific AADs for AF patients with different comorbidities and clinical presentations

Participating learners were stratified into 2 tiers, based upon needs identified for each individual learner. Individualization of the curriculum was conducted using a computerized platform (RealCME) that identifies learning gaps (tier 1) and then directs learners to tailored educational activities (tier 2) based on their performance in tier 1.

2.1.1. Tier 1 curricula

The tier 1 activities consisted of 2 live regional case‐based symposia (Boston on October 9, 2011, or Rosemont on December 10, 2011) and 3 online office‐visit interactions (Program A, B, C) with virtual patients. The online activities were launched on December 27, 2011. Each of the separate patient cases focused on specific predetermined learning objectives. Evaluation of participant deficits based on designated parameters (ie, learning objectives) was conducted during the tier 1 activities and specific learning gaps were identified. Learners then received emails directing them to continue their individualized CME pathways with recommended tier 2 activities.

2.1.2. Tier 2 curricula

The individualized tier 2 activities were created to address the specific learning gaps identified in tier 1 and consisted of 2 online activities: a simulated office visit with a virtual patient (program D) addressing safety/efficacy guidelines and a virtual grand rounds activity on specific treatment approaches. Tier 2 activities were made available on January 20, 2012. Learning performance was assessed upon completion of the final activity. Approximately 8 weeks following tier 2 activities, learners were invited to participate in a postcurriculum assessment (PCA) designed to evaluate learner knowledge retention. The PCA was made available on February 14, 2012.

2.2. Assessment of learning and performance

2.2.1. Learner Assessment Platform analysis

This study employed a multitiered assessment‐based model of analysis based on learning objective performance. Change in learner performance (based on the RealIndex) was measured for each learner track, from tier 1 to tier 2. Learners were assessed based on cohort groupings defined by whether or not they completed the Learner Assessment Platform (LAP) (LAP cohort) (tier 1 followed by the recommended tier 2 activity/activities), or did not complete the LAP (completed multiple‐activities that were not according to the LAP recommendations, or completed only 1 activity (in either tier 1 or tier 2) (non‐LAP cohort).

2.2.2. Learner domains

Learning was assessed using criteria from Donald E. Moore Jr.6 Pretests and post‐tests were administered for each activity. All pretest questions were paired with the identical question in the post‐test. Each activity assessment contained a variety of question types that were focused on 4 specific learner domains (knowledge, competence, confidence, and performance). The knowledge and competence assessment questions were mapped to specific learning objectives. The confidence and practice questions were assessed using a Likert scale ranging from 1 = never to 5 = always. The pretest and post‐test questions were matched, paired, and tagged by question type, learning objective, and subject area in the RealMeasure platform. The 4 learner‐domains were measured using activity‐specific questions (paired and delivered at both pretest and post‐test for each activity) that were averaged to generate cumulative scores across all activities of the curriculum.

2.2.3. Performance assessment

Performance was based on the RealIndex, which is a composite score based on multidimensional situation‐based questions that address specific learning objectives. Participants were presented with multidimensional situation‐based questions where statements related to the clinical scenario were answered as either “consistent” or “inconsistent” with the physician's own current practice approach. These questions assessed learners' knowledge of the latest published data, diagnostic tools, pertinent clinical guidelines, and their ability to actively apply this knowledge to the scenario presented. The performance assessment was administered prior to the first activity (baseline), following each activity, and in the follow‐up assessment. During each subsequent administration of the RealIndex questions, learners are given the opportunity to refine their responses based on their progressive learning. Because the identical case scenario was utilized in all activities, performance changes can be monitored while learners are progressing through the multiple activities of a curriculum. The RealIndex model is designed to align the progression of a learner's RealIndex score from baseline through the final activity with the changes that the learner is making in their practice approaches.

2.3. Statistical analysis

The data were analyzed using the Statistical Package for Social Sciences version 19.0 (IBM Cor., Armonk, NY). The data were first arrayed using frequencies. Assessment questions were evaluated by pretest and post‐test comparisons employing a paired statistical method. Paired and unpaired t tests were used to assess the differences between the mean responses of the participants; differences were considered significant for P values reaching 0.05 or less. Paired t test calculations were only conducted on matched pairs of participant scores. Analysis of variance (ANOVA) was carried out to assess differences between groups of learners (cohorts).

3. RESULTS

3.1. Overall participation

In total, there were 11 627 activity interactions, and high engagement levels were observed throughout the curriculum (Table 1). Completion rates in all of the activities (Table 1) were consistently high (>63%), and a high proportion of learners participated in multiple activities. Fifty‐nine percent (N = 3686) of unique learners started 1 activity, and 41% participated in multiple activities. An average of 77% of the participants who started an enduring activity completed the activity's post‐test, and 59% claimed certificates (Table 1).

Table 1.

Physician participation in the continuing medical education curriculum

Activity	Started Activity, No.	Completed Post‐test		Certificates Issued
		No.	%	No.	%
Tier 1
Live event–Boston	66	58	97%	—	—
Live event–Rosemont	66	65	100%	—	—
Program A	3516	2223	63%	1904	54%
Program B	1787	1248	70%	1132	63%
Program C	1934	1350	70%	1155	60%
Tier 2
Program D	1958	1332	68%	1226	63%
Antiarrhythmic therapy in AF	1972	1240	68%	1251	63%
All activities	11 299	7616	77%	6668	59%

Abbreviations: AF, atrial fibrillation.

3.2. LAP pathway participation

Of the total population, 41% of learners participated in multiple activities; 1236 (20%) began 2, 564 (9%) began 3, 258 (4%) began 4, and 482 (8%) began 5 activities. Of the learners who completed at least 1 activity, 29% (n = 989) completed the recommended LAP pathway (tier 1 followed by the recommended tier 2 activity/activities). There are 2o different groups of non‐LAP learners; 1 group (16%, n = 533) completed multiple activities, but these activities were not according to the LAP recommended pathway, and the other group (56%, n = 1,899) completed only 1 activity (in either tier 1 or tier 2).

3.3. Learner domain improvements

Significant improvement in mean scores between pretests and post‐tests across the entire curriculum for all 4 specific learner domains (knowledge, competence, confidence, and practice) was observed (Figure 1). Mean scores improved by 48% (P < 0.0005), 78% (P < 0.0005), 21% (P < 0.0005), and 20% (P < 0.0005) for knowledge, competence, confidence, and practice, respectively. Individual learner score dispersion was observed for all domains, as indicated by the moderately elevated standard deviations.

Figure 1.

Relative improvements in mean scores across the learner domains. Abbreviations: SD, standard deviation.

3.4. Overall performance improvement across the curriculum

The average performance score of the total learner population showed a significant 28% increase (P < 0.0005) from 35% at baseline to 45% at the conclusion of their final activity (N = 3962, Figure 2), regardless of whether learners followed the guided CME pathway (LAP).

Figure 2.

Relative percent change in performance scores for all learners. Abbreviations: SD, standard deviation.

3.5. Comparison of the performance of LAP and non‐LAP learners

Based on whether learners followed the LAP recommended guided pathway, participants were separated into LAP and 2 different non‐LAP (completed ≥2 activities not to LAP recommendation or completed 1 activity) learner groups. All groups showed statistically significant gains (P < 0.0005) from baseline to their final intervention (Table 2); however, the LAP learners, who followed the guided pathway, demonstrated a greater improvement (41%) compared to 2 different groups of non‐LAP learners (23% and 26%, respectively). The baseline performance score of the LAP learners was slightly lower than that of the non‐LAP groups (not statistically significant); however, the LAP group earned a significantly higher (P = 0.003) final score compared to the non‐LAP groups.

Table 2.

Performance scores: LAP vs non‐LAP

Learner Group	No.	Baseline Average Score (SD)	Final Average Score (SD)	% Change	P Value
LAP	989	37% (21%)	53% (17%)	41	<0.0005
Non‐LAP (1 activity)	1899	39% (20%)	48% (18%)	23	<0.0005
Non‐LAP (2 activities)	533	40% (21%)	50% (18%)	26	<0.0005

Abbreviations: LAP, Learner Assessment Platform; SD, standard deviation.

3.6. Comparison of the performance retention of LAP and non‐LAP learners

The difference in performance scores between learners who followed a guided pathway (LAP) and those who did not (non‐LAP) also extended to the follow‐up assessment (Table 3). Three hundred ninety‐five participants started the PCA, and 190 completed it. LAP learners who participated in the PCA (n = 75) showed a significant (P < 0.0005) 36% net gain in the RealIndex score from baseline (34%) to the PCA (47%). A non–statistically significant modest decrease (4%) was measured from the LAP learners’ final activity (50%) to the PCA. The non‐LAP learners who completed the PCA (n = 106) also showed a statistically significant (P < 0.0005) gain of 18% from baseline (38%) to the PCA (45%), and a non–statistically significant modest decrease (2%) from their final activity; however, this improvement was comparatively lower (18%) than that of the LAP learners (36%). The average baseline score of the LAP learner group was lower than the non‐LAP group; however, the LAP group showed comparatively higher performance scores by their final activity assessment and the PCA.

Table 3.

Retention of performance scores: LAP vs non‐LAP

Parameter	No.	Average Score (SD)	Relative % Change From Previous Scores (P Value)	% Net Change PCA‐BL (P Value)
LAP learners
Baseline	75	34% (23%)	—	—
Final	75	50% (21%)	44% (<0.0005)	N/A
PCA	75	47% (22%)	−4% (0.075)	36% (<0.0005)
Non‐LAP learners
Baseline	106	38% (24%)	—	—
Final	106	46% (22%)	20% (<0.0005)	N/A
PCA	106	45% (17%)	−2% (0.541)	18% (<0.0005)

Abbreviations: BL, baseline; LAP, Learner Assessment Platform; N/A: not applicable; PCA, postcurriculum assessment.

4. DISCUSSION

CME programs focused on improving the timely diagnosis of AF and implementation of guideline recommendations to reduce morbidity and death are important in managing the increasing population of AF patients.1, 2, 3 The accredited curriculum developed in this study utilized a Learner Assessment Platform (LAP) that assessed learners’ proficiency in tier 1 activities and then directed them to individualized tier 2 activities based on identified learning gaps. Nine hundred eighty‐nine learners (29%) completed a LAP pathway, suggesting that learners are receptive to guided individualized pathway learning. The results from the RealIndex indicated that all participant tracks (LAP and non‐LAP) earned statistically significant gains from baseline; however, the LAP pathway learners achieved greater performance gains compared to non‐LAP learners, suggesting the LAP methodology may be effective in providing a cumulative and individualized learning experience. The results of the evaluation between the LAP and non‐LAP learner groups reflected a lack of perceived bias.

This study also demonstrates that guided individualized pathways of learning may be useful in closing knowledge gaps related to AF treatment among a wide diversity of healthcare providers. Both primary care providers (PCPs) and cardiologists who participated in the curriculum made significant gains across all learning domains; however, the results showed distinct differences between the 2 groups. Whereas cardiologists showed few score differences across both methodologies (LAP or non‐LAP), only the PCPs from the LAP learner group demonstrated substantially greater improvements on the RealIndex. The observation that the LAP group of PCPs achieved a comparable level of mastery as cardiologists further demonstrates the efficacy of the LAP methodology for targeting the specific needs of initially less proficient learners, and suggests that this may be an effective methodology to present specialized concepts and skills.

Additionally, results indicated that although slippage in average learner scores occurred from post‐test to the PCA, overall sustained net gains were measured from cumulative pretest scores across all learner domains. A further analysis of the PCA found that the LAP participants demonstrated a significantly greater net gain in performance and retention compared to non‐LAP participants. These PCA results provide evidence that this curriculum produced gains that were retained by learners 8 weeks after participation, and reinforces the value of guided individualized learning experiences.

Overall, LAP proved to be a successful methodology for substantially improving and sustaining learner performance and knowledge from low baseline levels. The data demonstrate that this methodology of guided learning has the potential to improve educational outcomes among both primary care and specialty groups in clinically challenging areas. Future studies focused on persistent knowledge gaps in treatment of AF would be beneficial. Results from this study found that treatment guidelines for patients presenting with comorbidities remained a challenge, suggesting that learners would continue to benefit from activities that address this particular area of patient management. Additionally, future studies are needed to determine whether these improved educational outcomes will directly translate to improved practice approaches and improvement in patient health outcomes. If clinicians apply the reported changes in practice behaviors concerning the treatment of AF patients, a positive impact on patient health outcomes may be expected.

5. CONCLUSION

Results from the evaluation of this ACCME accredited CME program confirm the value and impact of independent CME on physician knowledge acquisition, retention, and competence in determining appropriate patient care as demonstrated by case‐study decision making as a surrogate marker of performance. All physician learners showed significant improvements in knowledge; however, the LAP learners compared to the non‐LAP learners showed a greater improvement in knowledge, suggesting that this methodology of guided individualized education is effective. Moreover, the active participation rates in multiple components of the curriculum demonstrate learner engagement and the cumulative impact of the education with increased exposure specifically tailored to individual learners. Further studies are needed to determine whether these improvements in knowledge will be associated with a change in practice and improved AF patient health outcomes.

5.1. Conflicts of Interest

Brian Lee, PharmD, is the principle and Medical Director of Horizon CME, Inc., a medical education company that develops continuing education programs for healthcare professionals. Karyn D. Ruiz‐Cordell, PhD, is Chief Data Scientist of RealCME, an educational technology and analytics company. Steven Haimowitz, MD, is the Chief Executive Officer of RealCME, an educational technology and analytics company. Cara Williams, PharmD, is an employee of Horizon CME, Inc., a medical education company that develops continuing education programs for healthcare professionals. Bruce Stambler, MD, is the Director of Cardiac Arrhythmia Research and Education at Piedmont Heart Institute. Anthia Mandarakas is a Senior National Education Manager at Sanofi US and owns stock in Sanofi.

Lee BC, Ruiz‐Cordell KD, Haimowitz SM, Williams C, Stambler BS and Mandarakas A. Personalized, assessment‐based, and tiered medical education curriculum integrating treatment guidelines for atrial fibrillation, Clin Cardiol, 2017;40:455–460. 10.1002/clc.22676