Abstract
Background:
Case-based learning (CBL) is a distinct classroom-based teaching format. We compare learning and retention using a CBL teaching strategy vs simulation-based learning (SBL) on the topic of malignant hyperthermia.
Methods:
In this study, 54 anesthesia residents were assigned to either a CBL or SBL experience. All residents had prior simulation experience, and both groups received a pretest and a lecture on rare diseases with emphasis on malignant hyperthermia followed by a CBL or SBL session. Test questions were validated for face and construct validity. Postsession testing occurred on the same day and at 4 months.
Results:
Twenty-seven residents completed all components of the study. The CBL group had 10 residents, and the SBL group had 17 residents. Most residents (80%) had previous exposure to malignant hyperthermia education. ANOVA for repeated measures demonstrated superior learning and long-term retention in the CBL group. In addition, our cost analysis reveals the cost of SBL to be approximately 17 times more expensive per learner than CBL.
Conclusions:
We found that CBL promoted learning and long-term retention for the topic of malignant hyperthermia and it is a more affordable teaching method. Affordability and effectiveness evidence may guide some programs toward CBL, particularly if access to simulation is limited. The number of participants and full validation of the examination questions are limitations of the study. Further studies are required to validate the findings of this study.
Keywords: Education, case-based learning, simulation, malignant hyperthermia
Introduction
Medical educators have long recognized simulation-based learning (SBL) to be an effective method for developing clinical competency and skills compared with case-based learning (CBL) and problem-based learning (PBL).1–4 SBL involves a clinical scenario in a simulated environment and provides an experiential form of active, individual learning within a small group followed by a debriefing session to review key lessons with simulation faculty.
Unfortunately, comparative studies of SBL to CBL and PBL have been impaired by ill-defined or loosely interpreted standards of PBL and CBL instruction.3,5 Although PBL and CBL are framed around a case or challenging problem, PBL is a less defined process because it does not use prior preparation to prime learners and relies on a more open-ended, student-led technique in which the teacher is not an active facilitator or director of the conversation.6,7 In contrast to PBL, CBL focuses on clinical problem solving with guided inquiry and incorporates advanced preparation prior to small group sessions to promote an informed discussion with active facilitation, resulting in more targeted learning, learner satisfaction and efficient use of class time.5,6 In addition, CBL is a superior teaching method to PBL in comparative studies and for the development of clinical skills.6–10
In the current educational environment where active, hands-on experiential learning is touted as the best learning tool, it is important to recognize that more studies are finding alternative forms of active learning are effective for teaching, such as CBL.11 Demonstrating equality of outcome of classroom-based teaching techniques is important because simulation is not always available or if it is, not sustainable because of the cost and limitations of simulation-based teaching resources. CBL can be accomplished in simple, nontechnical educational settings. This study sought to determine the efficacy of CBL compared with SBL for teaching anesthesiology residents about the management of patients with malignant hyperthermia (MH).
Methods
We conducted a prospective study as a curriculum development project in October 2013. The Johns Hopkins Medicine Institutional Review Board determined this was not human subjects research. Anesthesiology residents from Johns Hopkins Hospital were assigned to 1 of 2 groups: CBL or SBL under a code name for blinding purposes. All residents had prior experience with simulation as part of their anesthesia education and training. One week prior to the project, both groups were emailed the same reading material for the study, which included a review paper on MH. CBL participants were instructed to read the selected material on MH prior to class in accordance with the learning theory of CBL. SBL participants received the same material and assignment notice without any additional instructions.
A 30-question pretest on MH was constructed by a faculty member with expertise in MH. The test questions were reviewed for face and construct validity by 4 MH experts from the Malignant Hyperthermia Association of the United States (MHAUS), who also have extensive background as educators in the field of anesthesiology. The pretest, consisting of 23 multiple choice and 7 true/false items, was administered to all residents prior to a lecture. In addition, each resident was asked to indicate their level of training, whether they read or discussed the assignments, and if they had prior clinical experience with MH.
Upon completion of the pretest, a lecture on rare and coexisting diseases, including MH, was given to all residents and then followed by either a CBL or SBL session. Each session was 1 hour in duration. The focus of the session was on recognition, such as relating increasing end tidal carbon dioxide in spite of increasing minute ventilation to MH, and management, such as administration of dantrolene. While only 1 instructor was required for the CBL session, 4 faculty were required to conduct 2 concurrent simulation sessions with an average of 4 residents per simulation room per hour. The simulation faculty provided the same talking points on MH as the CBL instructor. (Appendix) More simulation instructors were needed than CBL to accommodate all learners. There was 1 CBL instructor concurrently facilitating 2 groups of 11 to 12 anesthesia residents in each group.
After the SBL and CBL sessions, participants from both groups took a posttest (pretest with rearranged questions). A posttest 1 (repeat of posttest) was conducted at 4 months. The time frame of 4 months was chosen to ensure learning was being assessed and not memorization skills.
Statistical Analysis
We measured effect size to describe the magnitude of difference of the scores between the groups. The effect sizes were calculated using Cohen d and expressed as small (.2–.5), moderate (.5–.8), or large (> .8) differences.12 Due to group substructure and the repeated measures nature of the 3-test design, a linear mixed model or ANOVA of repeated measure model was constructed to test overall statistical significance of the differences between pretest, posttest, and posttest 1 tests and differences between the CBL and SBL groups.13 The analyses were performed using the R version 3.2.2 (R Foundation for Statistical Computing, Vienna, Austria).14 Statistical significance was set at P < .05, and all tests were 2-sided.
A Q-Q plot of the model showed an approximate straight line, so we could conclude that the normality assumption is satisfied and thus the repeated measures ANOVA is viable. We also plotted residuals for both groups, which indicated that repeated measure ANOVA is reasonable.15
Results
Fifty-four anesthesia residents were enrolled in the study. Fifty-one residents took the pretest and attended the lecture. Forty-four residents were assigned to either SBL or CBL. Twenty-seven residents completed the study. (Figure 1) Of the 27 anesthesia residents who completed all 3 exams, 10 (37%) participated in the CBL group and 17 (63%) participated in the SBL group. Of the 10 residents participating in the CBL group, 5 (50%) were first year residents, 4 (40%) were second year residents, and 1 (10%) was a third year resident. Of the 17 residents participating in the SBL group, 8 (47%) were first year, 6 (36%) were second year, and 3 (17%) were third year residents. (Table 1) One hundred percent of CBL residents completed the required reading assignment, and 50% completed the additional reading assignment. Fifty-four percent of all SBL residents read some material from the reading list, and less than 30% completed the reading assignment required of the CBL group. Approximately 80% of members of each group reported some exposure to MH in the form of class discussion, patient care, or personal experience. Twenty-seven of the original 54 residents were unable to complete all components of the study because of clinical duties, postcall status, and time away from the residency.
Table 1.
CBL (N = 10) | SBL (N = 17) | P-Value | Previous MH Learning (N = 44) | |
---|---|---|---|---|
1st year; N (%) | 5 (50) | 8 (47) | *.86 | 19 (43) |
2nd year; N (%) | 4 (40) | 6 (36) | 15 (34) | |
3rd year; N (%) | 1 (10) | 3 (17) | 10 (23) |
Abbreviations: CBL, case-based learning group; SBL, simulation-based learning group.
* The p-value =.86 shows that there is no significant difference among 1st, 2nd and 3rd year anesthesia residents across CBL vs SBL.
Analysis of test results showed significant improvement of resident scores on both the posttest and posttest 1 tests, indicating that all residents learned. (Figure 2) Residents belonging to the CBL group had higher average test scores. (Figure 2) The overall average score using all tests (pretest, posttest, and posttest 1) in the CBL group was 24.0 compared with 21.64 in the SBL group (P < .001). However, our analysis of scores within groups showed a higher average baseline score of 21.9 in the CBL group compared with average 19.6 in the SBL group, resulting in no difference in change of scores between groups (P = .06). The magnitude of the difference between the groups (CBL vs SBL) as measured by effect size was 1.09 for the pretest, 1.45 for the posttest, and 1.47 for posttest 1. The effect size demonstrated the difference between scores associated with CBL vs SBL group across all tests was significantly large.
The ANOVA table of repeated measures model demonstrated a significant difference between CBL and SBL groups. (Table 2) The residents belonging to the CBL group had scores significantly higher as compared with the SBL group. (P = .007) The performance of residents on the posttest and posttest 1 tests significantly improved in both groups compared with the pretest. (P < .001) There was no significant difference in the performance of residents on posttest vs posttest 1 (P = .55).
Table 2.
Coefficient (Pr) | P-Value | |
---|---|---|
Group (CBL vs SBL) | −2.25 | .007 |
Pretest vs Posttest | 2.88 | < .001 |
Pretest vs Posttest 1 | 3.22 | < .001 |
Posttest vs Posttest 1 | −.33 | .55 |
Abbreviations: CBL, case-based learning group; SBL, simulation-based learning group.
a ANOVA testing demonstrated residents belonging to the CBL group had significantly higher scores, (P value = .007) and the performance of residents on the posttest and posttest 1 significantly improved in both groups compared with the pretest. (P < .001)
Analysis of resources for CBL vs SBL reveals significant cost differences based on the total number of learners. One faculty concurrently facilitated 2 groups of 11 and 12 residents each during the 1-hour CBL session, although only 20 residents completed the posttest. Four faculty taught concurrent 1-hour SBL sessions with 24 total residents over 3 hours with 2 faculty and approximately 4 residents per simulation room. Simulation center costs are $125 per hour per room. In our case, the total cost for 2 simulation rooms for 3 hours was $750. Accounting for faculty and simulation costs is shown in Table 3 and yields a cost of $5.60 per resident for CBL instruction vs $95.00 per resident for SBL instruction.
Table 3.
Teaching Method | No. Residents Taughta | No. Faculty | Faculty Costb | Simulation Center Cost | Total Cost | Cost Per Resident |
---|---|---|---|---|---|---|
CBL | 23 | 1 | $128 | 0 | $128 | $5.60 |
SBL | 24 | 4 | $1536 | $750 | $2286 | $95.00 |
Abbreviations: CBL, case-based learning group; SBL, simulation-based learning group.
a Residents taught = actual residents that attended the teaching session, based on original randomization to case-based learning vs simulation-based learning groups. These numbers are different from final study participant numbers that are based on the number of residents who completed all study tests.
b Faculty cost based on 2017 Bureau of Labor Statistics average hourly wage of $128 for anesthesiologists.
Discussion
This study demonstrates that learning and retention in the CBL group was superior to the SBL group. The difference in test scores within and between groups appeared to be representative of each group. Comparison of CBL postscores vs SBL postscores, while taking into account CBL prescores and SBL prescores, found there was no difference between the CBL and SBL group. Both teaching strategies within their respective groups demonstrated equal effectiveness in promoting retention based on the posttest and the final 4-month posttest 1 test results. The retention scores are higher than the expected exponential decline in memory retention of 40% at 20 minutes and 80% at 31 days based on the The Forgetting Curve of Dr. Ebbinghaus.17 Analysis of test scores between the 2 groups demonstrated a significant difference in learning retention for the CBL group, however their initial test scores were higher. The opportunity for spaced learning with a reading assignment and CBL interspersed with testing as another form of learning opportunity may have contributed to long-term retention through reconsolidation of stored information.
Our study is limited by the use of newly created examination questions. They were developed by an MH expert and evaluated for face and construct validity by 4 MHAUS experts, but they were not validated further. No predetermined passing score, nor critical questions had to be answered correctly to establish learning. Completion of the reading assignment was more common among the CBL group residents, which may have affected baseline testing, because of the inherent bias regarding preparation. Testing prior to reading assignments to establish a baseline assessment may have helped to more clearly define progression of learning, however there was concern for introducing bias by alerting residents to future testing on MH.
Analysis of test performance in each group suggests the higher baseline score of the CBL group may have influenced subsequent testing outcomes. This advantage in baseline testing may reflect the higher compliance with the preparatory reading assigned to the CBL group. The study was also limited by the presence of multiple SBL instructors vs 1 CBL instructor. Even though the same talking points were provided to the SBL instructors and the CBL instructor, it is difficult to enforce the inclusion or exclusion of talking points by the SBL faculty and individual group characteristics or instructor bias that might introduce differences in delivered content. In addition, the number of residents able to complete all study components was small and a crossover study was not feasible because of time constraints, resources, and concerns for learner fatigue and habituation to tests. Time constraints leading to separation of testing periods and CBL and SBL sessions contributed to the attrition rate throughout the day.
Overall, both groups showed evidence of learning, however future studies are needed to determine the best instructional strategies to enhance retention for clinical application and patient outcomes. Learning represents more than rote memorization; it is a process of encoding, consolidation, and recall.17 Simulation teaching for the acquisition of clinical skills and performance improvement remains an integral part of anesthesia training, but retention and transfer tests are needed to assess the use of simulation to teach relevant skills in the clinical setting, such as surgical skills.18
Demonstrating the efficacy of CBL vs SBL is important because simulation is not always available or cost may prohibit its routine use. Our cost of simulation was 17 times higher than the CBL teaching strategy; however, there are likely topics or skills for which the use of simulation is more effective than classroom-based teaching strategies. The ability to critically evaluate outcomes of learning and choose the most cost-effective instructional method per topic is needed. This study suggests for the purposes of improving knowledge about MH as measured by a written examination, CBL is the most cost-effective teaching method.
Conclusions
We present preliminary evidence of the value of CBL compared with SBL. This study demonstrates that based on a written examination assessment, teaching anesthesia residents about MH and care of MH patients can be accomplished in the classroom as well as in the simulation-learning environment. Based on our findings and the higher expense associated with SBL, we conclude that CBL is a cost-effective alternative to SBL for this topic. Future studies should seek to determine if both methods are effective and/or equivalent for teaching bedside management of the MH patient.
Acknowledgments
We wish to recognize the assistance of John H. Shatzer, PhD, Associate Professor of Medical Education at The Johns Hopkins University School of Education, Ms. Shannon Poling and Julianne Perretta, Simulation Educators at The Johns Hopkins Medicine Simulation Center, and Ms. Jacqueline Cross, Program Coordinator, and Ms. Andy Collins, Administrative Coordinator, for the Anesthesiology Residency Office at The Johns Hopkins Hospital.
Funding: This work was supported by the Clinical Research Core of the Department of Anesthesiology and Critical Care Medicine, Johns Hopkins University.
Appendix
Appendix. Malignant Hyperthermia Talking Points for Instructors of Simulation-based Learning and Case-based Learning Groups
Malignant hyperthermia (MH) – what it is
MH pathophysiology intracellular mechanism - unregulated release of calcium
Genetics of MH, reduced penetrance variable expressivity
Anesthetic triggering agents
Clinical presentation under anesthesia
Masseter muscle rigidity and malignant hyperthermia
Recognition of classic blood gas for MH
Rhabdomyolysis, hyperkalemia, acidosis, cardiac arrest
Calcium, calcium channel blockers
Dantrolene sodium mechanism of action, adverse/side effects, lack of response to nondepolarizing muscle relaxants
Atypical presentation, late presentation of MH in postoperative period
Management of MH; dantrolene - recommendations for stocking and mixing
Post MH crisis treatment
Recrudescence of malignant hyperthermia
Testing for MH: Caffeine halothane contracture test (CHCT) and genetic, costs
Eligibility criteria for genetic testing
Neuroleptic malignant syndrome, serotonin syndrome
Associated conditions of MH: Central core disease, King-Denborough syndrome
Preparation for MH susceptible patients: anesthesia machine, drugs, equipment
MH, pregnancy, and genetic inheritance
Awake MH susceptible patients
Footnotes
- Tae W. Kim helped design, execute, and write the manuscript.
- Sarabdeep Singh helped conduct analysis and interpretation of data and helped write the manuscript
- Christina Miller helped design, execute, and write the manuscript.
- Shivani Patel helped design, execute, and write the manuscript.
- Rahul Koka helped design, execute, and write the manuscript.
- Adam Schiavi helped design, execute, and write the manuscript.
- Deborah Schwengel helped design, execute, and write the manuscript.
References
- 1.Steadman RH, Coates WC, Huang YM et al. Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med. 2006 Jan;34(1):151–7. doi: 10.1097/01.ccm.0000190619.42013.94. [DOI] [PubMed] [Google Scholar]
- 2.Seybert AL, Smithburger PL, Kobulinsky LR, Kane-Gill SL. Simulation-based learning versus problem-based learning in an acute care pharmacotherapy course. Simul Healthc. 2012 Jun;7(3):162–5. doi: 10.1097/SIH.0b013e31825159e3. [DOI] [PubMed] [Google Scholar]
- 3.Littlewood KE, Shilling AM, Stemland CJ et al. High-fidelity simulation is superior to case-based discussion in teaching the management of shock. Med Teach. 2013;35(3):e1003–10. doi: 10.3109/0142159X.2012.733043. [DOI] [PubMed] [Google Scholar]
- 4.Lee Chin K, Ling Yap Y, Leng Lee W, Chang Soh Y. Comparing effectiveness of high-fidelity human patient simulation vs. case-based learning in pharmacy education. Am J Pharm Educ. 2014 Oct 15;78(8):153. doi: 10.5688/ajpe788153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Schwartz LR, Fernandez R, Kouyoumjian SR, Jones KA, Compton S. A randomized comparison trial of case-based learning versus human patient simulation in medical student education. Acad Emerg Med. 2007 Feb;14(2):130–7. doi: 10.1197/j.aem.2006.09.052. [DOI] [PubMed] [Google Scholar]
- 6.Nadershahi NA, Bender DJ, Beck L, Lyon C, Blaseio A. An Overview of Case-based and Problem-based Learning Methodologies for Dental Education. J Dent Educ. 2013 Oct;77(10):1300–5. [PubMed] [Google Scholar]
- 7.Srinivasan M, Wilkes M, Stevenson F et al. Comparing problem-based learning with case-based learning: effects of a major curricular shift at two institutions. Acad Med. 2007 Jan;82(1):74–82. doi: 10.1097/01.ACM.0000249963.93776.aa. [DOI] [PubMed] [Google Scholar]
- 8.Thistlethwaite JE, Davies D, Ekeocha S et al. The effectiveness of Case-based Learning in health professional education. A BEME systematic review: BEME Guide No. 23. Med Teach. 2012;34(6):e421–44. doi: 10.3109/0142159X.2012.680939. [DOI] [PubMed] [Google Scholar]
- 9.Aluisio A, Daniel P, Grock A et al. Case-based Learning Outperformed Simulation Exercises in Disaster Preparedness Education Among Nursing Trainees in India: A Randomized Controlled Trial. Prehosp Disaster Med. 2016 Oct;31(5):516–23. doi: 10.1017/S1049023X16000789. [DOI] [PubMed] [Google Scholar]
- 10.Raurell-Torredà M, Olivet-Pujol J, Romero-Collado À et al. Case-based learning and simulation: useful tools to enhance nurses' education? Non-randomized controlled trial. J Nurs Scholarsh. 2015 Jan;47(1):34–42. doi: 10.1111/jnu.12113. [DOI] [PubMed] [Google Scholar]
- 11.Couto TB, Farhat SC, Geis GL, Olsen O, Schvartsman C. High-fidelity simulation versus case-based discussion for teaching medical students in Brazil about pediatric emergencies. Clinics (Sao Paulo) 2015 Jun;70(6):393–9. doi: 10.6061/clinics/2015(06)02. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kelley K, Preacher KJ. On Effect Size. Psychol Methods. 2012 Jun;17(2):137–52. doi: 10.1037/a0028086. [DOI] [PubMed] [Google Scholar]
- 13.Laird NM, Ware JH. Random-effects models for longitudinal data. Biometrics. 1982 Dec;38(4):963–74. [PubMed] [Google Scholar]
- 14.The R Foundation The R Project for Statistical Computing. http://www.R-project.org/ Accessed April 7, 2016.
- 15.Chen D, Peace KE. Clinical Trial Data Analysis Using R. CRC Press; Boca Raton, FL: 2011. pp. 121–149. [Google Scholar]
- 16.US Bureau of Labor Statistics Occupational Employment and Wages, May 2017: 29-1061 Anesthesiologists. https://www.bls.gov/oes/2017/may/oes291061.htm Accessed March 3, 2019.
- 17.Stahl SM, Davis RL, Kim DH et al. Play it Again: The Master Psychopharmacology Program as an Example of Interval Learning in Bite-Sized Portions. CNS Spectr. 2010 Aug;15(8):491–504. doi: 10.1017/s1092852900000444. [DOI] [PubMed] [Google Scholar]
- 18.Moulton CA, Dubrowski A, MacRae H et al. Teaching surgical skills: what kind of practice makes perfect?: a randomized, controlled trial. Ann Surg. 2006 Sep;244(3):400–9. doi: 10.1097/01.sla.0000234808.85789.6a. [DOI] [PMC free article] [PubMed] [Google Scholar]