Abstract
Background:
Point-of-care ultrasonography (PoCUS) provides real-time, dynamic clinical evidence for providers to make potentially lifesaving medical decisions; however, these tools cannot be used effectively without appropriate training. Although there is always the option of traditional didactic methods, there has been a recent trend toward a “reverse classroom” web-based model using online e-learning modules. Our objective was to collect pilot data that would justify a future randomized controlled trial, comparing traditional didactics to an e-learning PoCUS curriculum for lung ultrasonography (LUS) and the focused assessment with sonography in trauma (FAST) exam.
Methods:
Anesthesiology interns, residents (CA 1–3), and fellow trainees enrolled in a LUS and FAST exam course and were randomized to receive didactic lectures or e-learning. Trainees completed knowledge pre- and posttests. Surveys were administered to gauge learning satisfaction. All trainees completed a hands-on-training (HOT) workshop. Image acquisition was assessed through practical tests before HOT, immediately after HOT, and 5 months later.
Results:
Eighteen trainees completed the study. There was no evidence of a difference in change in LUS knowledge test score from baseline to posttest between the e-learning and didactic groups (difference in median percentage point change [95 % CI]: 6.6 [−10.0, 23.2]; P = .411). There was no evidence of a difference in LUS knowledge posttest scores (difference in median percentage points [95% CI]: −0.9 [−4.8, 3.0]; P = .629), FAST knowledge posttest score (0 [incalculable]; P = .999), or post-HOT practical test score (−4.2 [−24.6, 16.3]; P = .672) between groups. There was no evidence of a difference in degree of satisfaction with learning experience between groups (odds ratios [95% CI]: 1.75 [0.31, 9.94]; P = .528).
Conclusions:
There was no evidence of a difference between the e-learning and traditional didactic groups in learning or satisfaction outcomes. These results justify establishing an adequately powered, randomized controlled trial assessing the noninferiority of e-learning to traditional didactics for teaching LUS and FAST.
Keywords: point-of-care ultrasonography, anesthesiology education, curriculum, teaching methods
Introduction
Point-of-care ultrasonography (PoCUS) provides real-time, dynamic clinical evidence for providers to make potentially lifesaving medical decisions. PoCUS is rapidly increasing in popularity and has been described as the “stethoscope of the 21st century.”1 Recently, PoCUS has expanded into acute care specialties such as anesthesiology. Lung ultrasound (LUS), an essential PoCUS skill, is more reliable than chest x-ray for ruling out pneumothorax,2 which is a potential complication of anesthetic procedures such as gaining central venous access as well as performing peripheral nerve blocks. Pneumothorax is notably difficult to diagnose in an anesthetized patient, but LUS can be reliably used to quantify pneumothorax.3 LUS is also very accurate for differentiating among causes of respiratory compromise due to interstitial syndromes such as congestive heart failure (CHF), acute lung injury (ALI), or acute respiratory distress syndrome (ARDS).4 The focus assessment with sonography for trauma (FAST) exam is a commonplace PoCUS skill utilized in the emergency room to screen for the presence of intraperitoneal free fluid following trauma. The FAST exam can detect free fluid with high sensitivity and specificity (0.64 to 0.98 and 0.86 to 1.00, respectively).5 The FAST exam is a helpful tool for evaluating pericardial effusions as well as other obvious cardiac pathology.6 The FAST exam has also recently been shown as meaningful in the perioperative setting to assess for fluid extravasation and potential abdominal compartment syndrome following hip arthroscopy.7
Despite these benefits, PoCUS cannot be utilized effectively without proper training in ultrasound image acquisition and interpretation. As PoCUS use increases in the perioperative setting, there has been a call to action to provide a fundamentals of perioperative ultrasound training experience for anesthesia residents and fellows.8,9 When implemented appropriately and broadly, a systemic education program can lead to improved PoCUS skills among anesthesiology trainees.10 Although medical faculties and critical care societies have developed comprehensive ultrasonography curriculums,8,11 one challenge of providing training is determining the best teaching modality. Didactics require a rigid time commitment and allocation of faculty resources, while a “reverse classroom” web-based model using online e-learning modules allows for more scheduling flexibility without sacrificing transmission and progression of information involved in classroom-based learning. However, an important question is whether this material can be taught effectively to anesthesiology trainees with either a didactic or e-learning method. Given the challenge of providing repeated didactic lectures, which requires allocating faculty resources and time, e-learning offers a more efficient means to provide this training. There is evidence that e-learning and related simulated training are efficient in providing PoCUS training. After a 5-minute online training tutorial, anesthesia residents and faculty could correctly rule out pneumothorax with 86.6% sensitivity and 85.6% specificity.12 Simulator training improved comprehension and practical performance of transthoracic echocardiography compared with didactic training.13 At the Hospital for Special Surgery (HSS), e-learning was effective in improving comprehension in basic focus-assessed transthoracic echocardiography (FATE), demonstrated by a median (quartile 1 [Q1], quartile 3 [Q3]) 40.4 (22.8, 56.3) percentage point increase in scores between an online pre- and posttest (P < .001).14
To the authors' knowledge, there has not yet been an assessment of teaching a PoCUS curriculum for both LUS and the FAST exams for anesthesiology trainees. Through this pilot study, we randomized trainees to either traditional didactics or e-learning models and measured learning and satisfaction outcomes. We aimed to use our results to justify a large definitive trial comparing traditional didactics to e-learning in the context of PoCUS training.
Methods
Study Overview
This randomized trial was approved by the Institutional Review Board (IRB #2015-482) at HSS. Written informed consent was waived because consent was implied when the anesthesiology trainees registered for the study. The study was conducted at HSS, a Weill Cornell Medical School (WCMS)-affiliated teaching hospital that specializes in musculoskeletal disease.
Study Participants
The study population comprised of anesthesiology interns, residents in clinical anesthesia (CA) years 1–3, and HSS regional anesthesiology and acute pain fellows. Trainees were randomized into one of two teaching methods, classroom or e-learning, such that they were equally distributed based on the level of experience. SAS software's PROC SURVEYSELECT was used to randomize trainees to either the e-learning or didactic group, and to Hands-On-Training (HOT) workshop groups with stratification based on the level of experience (junior vs senior). For the study, junior experience level included interns and CA year 1 and 2 residents. Senior experience level included CA year 3 residents and HSS fellows.
Study Design
The anesthesia trainees were taught how to perform and interpret LUS and the FAST exam by first providing didactic knowledge followed by directed HOT in the skills. For the knowledge-based portion of the course, the e-learning group and classroom group received training separately. The e-learning group had access to the Ultrasound of Airway, Breathing, Circulation and Dolor (USabcd. org) Basic Lung and FAST e-learning courses 2 weeks before the HOT course. USabcd. org is a commercially available website that offers point-of-care ultrasound courses that are focused, structured, and designed to improve diagnosis and patient treatment in critical, emergency, and perioperative care. E-learning courses utilized for this study through USabcd.org are integrated into ultrasonography courses provided by the American Society of Regional Anesthesia and Pain Medicine (ASRA) as well as in training courses offered worldwide. The e-learning courses consist of a pretest, a total of 14 interactive educational modules (1) Basic Lung Ultrasound9: introduction, equipment, and skills, examination technique, anatomy/sonoanatomy, pathology, pneumothorax, pleural effusion, interstitial syndrome and summary; (2) FAST5: introduction, equipment, and skills, anatomy and sonoanatomy, pathology and summary containing numbered quizzes throughout and a posttest. The pre- and posttests consist of the same questions to gauge improvement in understanding following completion of the e-learning or didactic session. The FAST pre- and posttests were 18 questions long and addressed relevant normal sonoanatomy as well as how to identify free fluid within the peritoneum and pericardium. The LUS preand posttests were 57 questions long and addressed relevant normal sonoanatomy as well as how to identity a pneumothorax, interstitial syndrome, and pleural effusions. The didactic group's pre- and posttests were graded on a separate date, and neither group was given an opportunity to review the answers to the tests. Trainees in the e-learning group were instructed to complete all three components of the Basic Lung and FAST e-learning courses before the HOT workshop. There was no time requirement for each module, but e-learning trainees were allowed to revisit modules at their convenience. Trainees provided an estimate of how long they took to complete the e-learning course.
The classroom group received a traditional didactic lecture from two anesthesiologist members of the study staff (S.H. and J.N.) immediately before the HOT workshop. To ensure both groups were exposed to the identical material, the lecture consisted of the USabcd.org Basic Lung and FAST e-learning modules including quizzes presented in a didactic lecture format. The LUS and FAST lectures lasted approximately 1 hour each. Time for questions was included at the end of both lectures. A pre- and posttest, identical in format to the e-learning tests, were administered on paper with video of ultrasound images projected when appropriate. The pre- and posttests were graded using the same criteria as the e-learning program.
The HOT workshop took place on October 10, 2015. Before the HOT workshop, all trainees completed a survey on their level of training, experience with ultrasound, familiarity with LUS and the FAST exams, comfort level in accessing relevant views for LUS, and their overall satisfaction with the learning experience.
For the HOT workshop, trainees were randomized to one of four groups, equally distributed based on the level of training. Each subject received two sets of random six-digit codes. The first code was used to save images acquired by trainees, and the second code was used to blind the images from the assessors.
There were four workstations, each with one ultrasound machine, one healthy male volunteer, and an anesthesiologist faculty instructor. Two types of ultrasound machines were used: GE LogiqE (GE Healthcare, Chicago, IL) and Sonosite X-Porte (FUJIFILM SonoSite, Inc., Bothell, WA). The trainees were given a pre-HOT practical image acquisition test following completion of the didactic learning and before any hands-on instruction from the faculty. The practical test involved scanning the healthy male volunteer and acquiring images within a given timeframe. Thirty seconds were allotted to identify and obtain clips of the bat sign, lung sliding, lung pulse, B-lines, M-mode seashore sign, and diaphragm and pleura on the right and left sides. Forty-five seconds were allotted to identify and obtain clips of the right upper quadrant (RUQ) Morrison's pouch, left upper quadrant (LUQ) perisplenic space, pelvis longitudinal view, pelvis transverse, and subcostal 4-chamber view.
Following the practical pretest, the HOT workshop was split into four sessions: familiarity with the ultrasound machine knobs (depth, gain) and lung scanning, lung scanning reinforced with images of pathology, FAST exam scanning, and then FAST exam reinforced with images of pathology. Each session was preceded by a short introduction to the material covered. Followed each HOT session, the group of trainees rotated stations to gain experience with different ultrasound machines, models, and faculty instructors. Upon completion of the HOT course, all trainees were given a practical image acquisition posttest that was identical in structure to the pretest. Approximately 5 months following the HOT workshop, trainees were invited back to participate in a practical image acquisition retention test, identical to the tests administered pre-HOT and post-HOT. There was no knowledge test given during the 5-month follow-up period.
All practical test images were graded by two anesthesiologists blinded to the identity and group allocation of the residents/fellows. They were scored in a binary “Yes” or “No” manner and received 1 point for each of the images correctly visualized and 0 points if not correctly visualized. If there was more than a 2-point disagreement in the total score, that test was then graded by a third blinded anesthesiologist. The two highest total scores were averaged. The changes in practical test score from pre-HOT to post-HOT and retention performance were analyzed.
Statistical Analysis
Study data were collected and managed using REDCap electronic data capture tools hosted at HSS.15
Continuous variables are presented as means with standard deviations or medians with 1st and 3rd quartiles. Categorical variables are presented as counts and percentages. The degree of imbalance in baseline characteristics was assessed by calculating standardized differences. Standardized differences with the absolute value greater than 1.962 × (2/n)1/2 = 1.962 × (2/9)1/2 = 0.925 were taken to indicate more imbalance than would be expected by chance.15 Quantile regression was used to calculate differences in medians with 95% confidence intervals for continuous outcomes. Ordinal logistic regression was used to calculate odds ratios with 95% confidence intervals for ordinal categorical outcomes. Statistical analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC).
Results
A total of 20 trainees were enrolled and randomized into the study (10 per group). Two trainees dropped out because of unavailability for a total of 18 trainees participating (9 per group). There were 5 junior and 4 senior level trainees in the classroom group. There were 4 junior and 5 senior level trainees in the e-learning group. Seven trainees in the e-learning group and 3 trainees in the classroom group participated in the practical retention test (Figure 1). Baseline characteristics, including ultrasound experience and comfort performing ultrasound, are shown in Table 1. There was no notable difference in self-reported comfort and familiarity with the performance of LUS or the FAST exam between the e-learning group and classroom group.
Figure 1.
Study schematic.
Knowledge Test
Median (Q1, Q3) percentage point change in lung US knowledge test score from baseline to posttest was 21.0 (7.2, 25.5) in the classroom group and 27.6 (26.8, 39.2) in the e-learning group (difference in median percentage point change [95% CI]: 6.6 [−10.0, 23.2]; P = 0.411; Figure 2A). There was no evidence of a difference in change in FAST exam knowledge test score from baseline to posttest (difference in median percentage point change [95% CI]: −11.1 [−43.3, 21.1]; P = 0.476) or change in combined written exam test score from baseline to posttest (5.8 [−9.8, 21.4]; P = 0.441; Figure 2A). There was no difference in median knowledge posttest scores for Lung US (percentage point difference [95% CI]: −0.9 [−4.8, 3.0]; P = 0.629), FAST exam (0 [incalculable]; P = 0.999), or Combined (−0.5 [−4.5, 3.5]; P = 0.792; Figure 2B through D).
Figure 2.
Change in knowledge test (A) from pre- to posttest and knowledge test scores by type: (B) lung ultrasound, (C) FAST exam, and (D) combined lung ultrasound and FAST exam. FAST = focused assessment with sonography for trauma. Boxplots contain median (line), box with interquartile (IQR) range (25–75 percentile), whiskers with minimum and maximum values that lie within 1.5 times the IQR, mean (circle) and outliers (diamonds).
Practical Test
The e-learning group improved a median 20.8 (95% CI: −14.9, 56.6) more percentage points in practical test score from pre- to post-HOT than the traditional didactics group (P = 0.235). There was no evidence of a difference in median post-HOT practical test scores between groups (median percentage point difference [95% CI]: −4.2 [−24.6, 16.3]; P = 0.672)]; Figure 3B).
Figure 3.
Change in practical test scores (A) from pre-HOT to post-HOT and retention performance and (B) practical test scores pre-HOT, post-HOT and during retention performance. HOT = hands-on training.
Boxplots contain median (line), box with interquartile (IQR) range (25–75 percentile), whiskers with minimum and maximum values that lie within 1.5 times the IQR, mean (circle) and outliers (diamonds).
At 5 months post-HOT, the median (Q1, Q3) retention performance score was 75.0% (33.3, 79.2) in the classroom group (n = 3) and 58.3% (54.2, 87.5) in the e-learning group (n = 7). The median percentage point change in practical test scores from pre-HOT to retention performance was −4.2 (−25.0, 45.8) in the classroom group and 33.3 (12.5, 50.0) in the e-learning group (Figure 3B).
Survey Outcomes
When surveyed before the HOT, trainees randomized to the e-learning group had 5.38 (95% CI: 0.84, 34.35; P = 0.075) times the odds of being more satisfied with their group assignment than trainees randomized to the classroom group (Table 2). Upon completion, trainees in the e-learning group had a 1.75 (95% CI: 0.31, 9.94; P = 0.528) times the odds of being more satisfied with their overall learning experience. Trainees in the e-learning group reported spending a median (Q1, Q3) 2.5 (2,3) hours on the e-learning modules, while the classroom group received 2 hours of didactics.
Discussion
This study demonstrated that both e-learning and didactic teaching of a new point-of-care LUS and FAST exam along with hands-on training were associated with improvements in knowledge and practical performance, as well as high satisfaction. These pilot data justify establishment of an adequately powered, randomized controlled trial assessing the noninferiority of e-learning to traditional didactics on one or more metrics.
In a study similar to ours, Edrich and colleagues16 found that web-based LUS training for pneumothorax exclusion was not inferior to classroom-based training. A future randomized trial should test not only the capability to exclude pneumothorax but also identify all signs relevant to detection of pneumothorax, interstitial syndrome, and pleural effusion.
Another study created an online tutorial with comprehensive LUS modules; improvements in multiple-choice test scores from preto posttest were similar between medical students using the online tutorial and didactic lecture.17 However, students who received lectures also had the advantage of practical training, which may explain the difference in study results. PoCUS practical training using a human model or simulation mannequin improved image acquisition and content retention of cardiopulmonary function and volume status.13
In a study using a similar multimodality approach to teach the extended FAST exam, Platz and colleagues18 concluded that e-learning was as effective as classroom-learning. However, the varied trainee population, inability to randomize trainees, and lack of practical performance evaluation are barriers to interpretation. Therefore, a large, randomized trial is warranted.
There were several study limitations: First, the sample size was limited by the number of available anesthesia residents and fellows. Second, this study taught PoCUS skills on healthy models without pathology. Third, there was a low return rate for the practical retention test, especially among the classroom-based training group. This low return was due to the challenge of scheduling residents/fellows from separate institutions (HSS and WCMC) who were on different rotations than when taking the course 5 months prior. Finally, although it was not the intention of this study, it is a limitation that these results may not apply to more experienced anesthesiologists or physicians in other specialties, such as emergency medicine and surgery.
The goal of this study was to collect pilot data that would justify a large randomized controlled trial comparing e-learning and classroom-based learning models for LUS and the FAST exams. Given the call to action to train the next generation of anesthesiologists with these essential PoCUS skills,9 the results indicate that an adequately powered, noninferiority randomized controlled trial is warranted. An e-learning curriculum allows for a structured, standardized, and more widely accessible means of providing the knowledge-based aspect of PoCUS training. With the new focus on patient-centered care in academic medical centers and effective use of technology in medical education,19 the flexibility of e-learning may offer an additional advantage over the more traditional classroom-based learning while providing comparable training.
Footnotes
This study was funded by the Hospital for Special Surgery Anesthesiology Department Research and Education Fund, New York, NY. The REDCap electronic data capture tools are funded by the Clinical and Translational Science Center (New York, NY), grant number UL1 TR000457 from the National Center for Advancing Translational Sciences, National Institutes of Health, Bethesda, MD.
The authors do not declare any grants, sponsors, or funding sources that provided direct financial support to the research work contained in this manuscript.
References
- 1.Moore CL, Copel JA. Point-of-care ultrasonography. New Engl J Med. 2011;364(8):749–57. doi: 10.1056/NEJMra0909487. [DOI] [PubMed] [Google Scholar]
- 2.Ding W, Shen Y, Yang J et al. Diagnosis of pneumothorax by radiography and ultrasonography: a meta-analysis. Chest. 2011;140(4):859–66. doi: 10.1378/chest.10-2946. [DOI] [PubMed] [Google Scholar]
- 3.Mittal AK, Gupta N. Intraoperative lung ultrasound: a clinicodynamic perspective. J Anaesthesiol Clin Pharmacol. 2016;32(3):288–97. doi: 10.4103/0970-9185.188824. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lichtenstein DA, Meziere GA. Relevance of lung ultrasound in the diagnosis of acute respiratory failure: the BLUE protocol. Chest. 2008;134(1):117–25. doi: 10.1378/chest.07-2800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Körner M, Krötz MM, Degenhart C et al. Current role of emergency US in patients with major trauma. Radiographics. 2008;28(1):225–42. doi: 10.1148/rg.281075047. [DOI] [PubMed] [Google Scholar]
- 6.Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. 2009;28(25):3083–107. doi: 10.1002/sim.3697. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Haskins SC, Desai NA, Fields KG et al. Diagnosis of intraabdominal fluid extravasation after hip arthroscopy with point-of-care ultrasonography can identify patients at an increased risk for postoperative pain. Anesth Analg. 2017;124(3):791–99. doi: 10.1213/ANE.0000000000001435. [DOI] [PubMed] [Google Scholar]
- 8.Ramsingh D, Rinehart J, Kain Z et al. Impact assessment of perioperative point-of-care ultrasound training on anesthesiology residents. Anesthesiology. 2015;123(3):670–82. doi: 10.1097/ALN.0000000000000776. [DOI] [PubMed] [Google Scholar]
- 9.Mahmood F, Matyal R, Skubas N et al. Perioperative ultrasound training in anesthesiology: a call to action. Anesth Analg. 2016;122(6):1794–804. doi: 10.1213/ANE.0000000000001134. [DOI] [PubMed] [Google Scholar]
- 10.Bøtker MT, Vang ML, Grøfte T et al. Implementing point-of-care ultrasonography of the heart and lungs in an anesthesia department. Acta Anaesth Scand. 2017;61(2):156–65. doi: 10.1111/aas.12847. [DOI] [PubMed] [Google Scholar]
- 11.Fox J, Schlang J, Maldonado G et al. Proactive medicine: the “UCI 30,” an ultrasound-based clinical initiative from the University of California, Irvine. Acad Med. 2014;89(7):984–89. doi: 10.1097/ACM.0000000000000292. [DOI] [PubMed] [Google Scholar]
- 12.Krishnan S, Kuhl T, Ahmed W et al. Efficacy of an online education program for ultrasound diagnosis of pneumothorax. Anesthesiology. 2013;118(3):715–21. doi: 10.1097/ALN.0b013e31827f0979. [DOI] [PubMed] [Google Scholar]
- 13.Ramsingh D, Alexander B, Le K et al. Comparison of the didactic lecture with the simulation/model approach for the teaching of a novel perioperative ultrasound curriculum to anesthesiology residents. J Clin Anesth. 2014;26(6):443. doi: 10.1016/j.jclinane.2014.01.018. [DOI] [PubMed] [Google Scholar]
- 14.Haskins SC, Zhao J, Nejim JA et al. Evaluation of postgraduates following implementation of a focus assessed transthoracic echocardiography (FATE) training course – a pilot study. J Anesth Clin Res. 2017;8(9):763. doi: 10.4172/2155-6148.1000763. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Harris PA, Taylor R, Thielke R et al. Research electronic data capture (REDCap) – a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Edrich T, Stopfkuchen-Evans M, Scheiermann P et al. A comparison of web-based with traditional classroom-based training of lung ultrasound for the exclusion of pneumothorax. Anesth Analg. 2016;123(1):123–8. doi: 10.1213/ANE.0000000000001383. [DOI] [PubMed] [Google Scholar]
- 17.Cuca C, Scheiermann P, Hempel D Emerg Med Int. 2013. Assessment of a new e-learning system on thorax, trachea, and lung ultrasound; p. 145361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Platz E, Goldflam K, Mennicke M et al. Comparison of web-versus classroom-based basic ultrasonographic and EFAST training in 2 European hospitals. Ann Emerg Med. 2010;56(6):667. doi: 10.1016/j.annemergmed.2010.04.019. [DOI] [PubMed] [Google Scholar]
- 19.AAMC Institute for Improving Medical Education Effective use of education technology in medical education. Colloquium on educational technology: recommendations and guidelines for medical educators. Mar, 2007.