Assessment of EFAST training for final year medical students in emergency medicine clerkship

Abstract

Background

Extended Focused Assessment Sonography for Trauma (EFAST) is an important bedside tool for the management of multiple trauma patients. We aimed to evaluate the assessment of our EFAST education in the Emergency Medicine Clerkship (EMC) for final year medical students and the correlations of EFAST marks with other practical skill stations and the final multiple choice question (MCQ) exam marks.

Methods

Fifty-four final year medical students were trained on performing EFAST on human models during their 4-week clerkship. Students received an hour of didactic lecture, 4-hours practical sessions on human models, and completed a minimum of three EFAST examinations on trauma patients. Finally, the EFAST performance was evaluated on human models using a standard evaluation form during an Objective Structured Clinical Examination (OSCE). The marks of 51 students who completed the final exam were analyzed.

Results

The overall passing rate of the EFAST station was 88% (n: 45). EFAST station mark had significant weak correlations with other OSCE stations marks (p = 0.027, rho = 0.31), and with the final EMC mark (p = 0.032, rho = 0.3), but not with the final MCQ exam.

Conclusions

Final year medical students demonstrated effective EFAST learning as measured by their examination performance. One hour EFAST training and 4 -hours practice provide an acceptable level of skill for medical students. The EFAST final marks showed significant weak correlation with other OSCE station marks and final clerkship marks, but not with the final MCQ exam mark which assesses a different cognitive learning domain.

1. Introduction

It is important that undergraduate medical students have proper Emergency Medicine training. This is an international critical component of accreditation for medical schools.¹ Ultrasound is now an essential part of undergraduate medical education.^2,3 Emergency ultrasound education, especially for the senior undergraduate years, may help students to use bedside ultrasound in their future management of critically-ill patients. Ultrasound is an essential tool for the diagnosis, resuscitation, and critical care of trauma patients.^4,5 Extended Focused Assessment Sonography for Trauma (EFAST) is an important bedside tool for the management of multiple trauma patients. It is one of the main applications that should be taught.^2,3,6,7

Focused Assessment Sonography for Trauma, which has very high sensitivity in detecting intra-peritoneal fluid, is a relatively simple skill to learn.^8,9 EFAST training is an essential part of emergency ultrasound training which aims to familiarize the trainees with the basic physics of ultrasound, train them how to properly handle the ultrasound machine, and let them perform the exam in real life.10, 11, 12 It is based primarily on didactic and practical sessions.⁸ We have started FAST training in UAE since 2004 targeting mainly doctors who manage trauma patients.¹¹ We have realized over time that students are eager and capable of performing the ultrasound as they are more familiar with the new technology. Accordingly, we decided to include EFAST training in our undergraduate education since 2013.

Standards for undergraduate ultrasound training and assessment are, however, still in a developing phase and mainly based on didactic and practical sessions.⁸ Assessments for EFAST training include pre and post-test, and psychomotor skill evaluation.^3,7 Objective structured clinical examination (OSCE) assessments were used by some researchers to better understand students' performance on EFAST.⁶ To the best of our knowledge, the correlation between EFAST skills and other clinical skills (cardiopulmonary resuscitation, airway management, suturing, emergency case management) were not studied before. We aimed to evaluate the EFAST education at our Emergency Medicine Clerkship and its correlations with the final exam and other OSCE scores.

2. Methods

2.1. Study setting and population

Emergency Medicine Clerkship (EMC) of the College of Medicine and Health Sciences of United Arab Emirates University is a four-week rotation which is structured based on the curriculum recommendations of the Society for Academic Emergency Medicine and the International Federation for Emergency Medicine.^13,14 EFAST training is one of the clinical skills which is taught in the EMC curriculum in the final year in our College. This is a retrospective analysis of prospectively collected data of a cohort of students.

2.2. Participants

Fifty-four final year medical students, who have no prior ultrasound training and hands-on experience, were trained on performing EFAST on human models in the clinical skills lab during the academic year of 2013–2014 (August 2013 to June 2014) by a single tutor having 12-years’ experience in Point-of-Care Ultrasound.

2.3. Ultrasound equipment

Siemens Acuson P300 ultrasound machine with a convex transducer (low frequency, 2–5 MHz) and a linear transducer (high frequency, 8–18 MHz) was used for training and assessment. An additional machine was supplied to support the practical sessions.

2.4. Teaching sessions

Students received 1-hour didactic lecture which included basic physics, knobology, artifacts, EFAST technique followed by a 2-hours practical session with tutor supervision on human models. EFAST included three intra-peritoneal windows (hepatorenal space, splenorenal space, pelvic space), four intrathoracic windows (right and left pneumothorax investigation, right and left intra-thoracic free fluid investigation), and a subcostal pericardial window. Students were exposed to the application, and to the normal and pathological views during the didactic session. In the practical session, they learned how to achieve an acceptable view for each sonographic window.

Each group consisted of 9–13 students. In the practical sessions, the group was divided into two subgroups. Each subgroup was trained and supervised by an instructor. During the 2- hours practical session, each student practiced for a period of 17–30  minutes on human models.

The students were asked to complete three EFAST examinations on multiple trauma patients in the Emergency Department during their clerkship period under the direct supervision of an attending emergency physician. Finally, the students had another 2-hours free practice on human models during the last week of rotation before the clerkship exams.

2.5. Assessment

EFAST performance of the students was evaluated on human models in a station as a part of the final clerkship OSCE. Assessment of all groups was done by two core faculty members of the Emergency Medicine residency program. Both have experience of ultrasound in clinical practice and were not involved in the EFAST teaching sessions. The examination form includes all validated steps of a standard EFAST examination. Students were examined on the technique and achieved image quality in the EFAST station. Transducer selection and orientation, subcostal pericardial window, hepato-renal space (Morrison's pouch), right pleural space for effusion, spleno-renal space, left pleural space for effusion, pelvic transverse and sagittal windows, left and right chest view with M-mode application for pneumothorax were evaluated in standardized forms. Students' performances were marked as one of three options (completely done, partially done, not attempted or wrongly done). Pre-OSCE instructive meetings with examiners of all OSCE stations were done 30  minutes before the OSCE. The mark selection on the evaluation forms was explained to the examiners to assure internal consistency. OSCE examinations were started at 08:30 in all groups. There were eleven active OSCE stations covering the Emergency Medicine Clerkship. Four stations included EFAST, basic and advanced cardiac life supports skills including cardiopulmonary resuscitation, airway management, and suturing. Seven stations were emergency case management stations. The total OSCE station marks were 25 out of 100. Each OSCE station was 2.27 marks. The passing mark for the clerkships as approved by the College and University Council was 75 out of 100. A score of 1.7 (75% of 2.27) was used as the EFAST station passing score (Fig. 1). The evaluation form includes a maximum of 10 points, and it equals 2.27 marks. The duration of the EFAST station was 6 min. The total EMC mark includes the following; OSCE (25 marks), the weekly multiple choice question (MCQ) exam (20 marks), the final MCQ exam (20 marks), case presentation (5 marks), patient and procedure encounters (20 marks), and supervisor overall evaluation (10 marks). EMC is given only in the final year of medical school. The students have no other structured Emergency Medicine rotation and no formal, standardized EFAST assessment in early years of our medical school.

Fig. 1.

Emergency medicine clerkship OSCE mark distribution and passing mark cut-off level.

2.6. Data analysis

We used crude marks of OSCE stations and exams. EFAST marks did not have a normal distribution (Kolmogorov-Smirnov test and Shapiro-Wilk tests, p < 0.0001 in both). Nonparametric statistical methods were used for comparison between groups and their correlations. The Mann-Whitney U test was used to compare the continuous data of two independent groups. Spearman's rank correlation was used to test the correlation between continuous variables. Probabilities of less than 0.05 were accepted as statistically significant. Data were analyzed using the Statistical Package for the Social Sciences (IBM-SPSS version 21, Chicago, Il).

3. Results

Fifty-one students (94.4%) completed the pre-OSCE teaching, practical sessions, and assessments of the course (male: 16, female: 35). Three students were not included in the analysis because they attended less than 2 -hours of hands-on session with faculty instructors (1 student), was unable to attend 2 -hours of free practical session (1 student), and unable to complete the needed three EFAST scans on trauma patients (1 student).

The mean mark for EFAST station was 1.94 (SD: 0.37) out of 2.27 (85%). The overall passing rate of the EFAST station was 88% (n: 45). The distribution of EFAST marks is shown in Fig. 2. They were skewed to the right.

Fig. 2.

Histogram of EFAST marks achieved by 51 medical students in the OSCE of Emergency Medicine Clerkship, UAE University, 2013–2014.

There was a significant weak correlation between EFAST station mark and other OSCE stations mark (EFAST excluded), (Spearman's rank correlation p = 0.027, rho = 0.31, Fig. 3). The EFAST station mark was not correlated with the final MCQ exam mark, Spearman's rank correlation p = 0.9, rho = − 0.02. The correlation between EFAST station mark and total EMC mark was also a significant weak correlation (Spearman's rank correlation, p = 0.032, rho = 0.3, Fig. 4).

Fig. 3.

The correlation between EFAST mark and other OSCE stations mark (p = 0.027, rho = 0.31, Spearman's Rank Correlation). Dark circles represent those who failed the EFAST station. Dashed lines are the pass mark.

Fig. 4.

Correlation between EFAST mark and final clerkship mark (p = 0.032, rho = 0.3, Spearman's Rank Correlation). Dark circles represent those who failed the EFAST station. Dashed lines are the pass mark.

4. Discussion

Our study has shown that the overall passing rate of the EFAST station of the final year undergraduate medical students in our college was 88%. There was a significant weak correlation between EFAST station mark and other OSCE stations marks but not with the final MCQ exam mark.

Pre and post-tests, and psychomotor skills evaluation are the main modalities to assess EFAST training.^3,7 EFAST has been previously included in OSCE examinations.⁶ Compared to our results, Wisniewski et al., showed a higher success rate with 96.5% having an adequate level of performance.³ In our study, students achieved an 88% passing rate on the EFAST OSCE station following 1 -hour didactic, 4-hours practice and a minimum of three EFAST exams on real trauma patients. The study of Wisniewski included 22 theoretical hours and 18 practical hours on ultrasound training over six days. It is, therefore, reasonable to expect a higher success rate as a result of longer exposure to didactic and practical sessions.¹⁵ Another study, which included 5-hours theoretical and practical sessions, showed that 85% of the students completed a full FAST scan at an adequate level of performance in less than 6  minutes, which is very similar to our results.⁶ The optimal EFAST training time for medical students is not yet determined. A recent report on EFAST training for medical students suggested a 4-hours period.¹⁶

EFAST needs visual and psychomotor coordination. Its skills can be learned in a short period of time because of its simplicity. Nevertheless, there are many factors that affect this performance. Past research in the mid 1970s¹⁷ found a better performance of male students on manual skills. Some studies, however, showed that female students' marks were significantly higher than male students' marks on skill stations such as history, physical examination, and counseling¹⁸ while other studies showed the opposite.¹⁹ Those results are inconstant and heterogeneous. We think that there are multiple factors affecting the impact of gender on practical skills.

The EFAST station mark was not correlated with the final MCQ exam mark. This is expected because each component assesses a different cognitive learning domain. Johnson reported that there was no significant relationship between MCQ exam and OSCE, which is similar to our results.²⁰ There was a significant weak correlation between EFAST station mark and other OSCE stations marks. Rogers et al. reported that the OSCE can be used as an effective performance evaluation tool for medical students.²¹ There was a significant weak correlation between EFAST station mark and the final mark of the clerkship in our study. Similarly, Merrick et al. reported that OSCE had a high correlation with the final grades.²² Although OSCE is mainly used in the postgraduate education of emergency medicine, efforts on high-quality OSCE for assessing undergraduate medical students are increasing.²³

5. Limitations

There are multiple limitations of our study. First, this is a single college study which includes all students in an academic year. The content of the EFAST lecture, practical hours, human models were the same for all groups over time. Second, 3 of 54 students did not complete the requirements and were not included in the analysis. Their exclusion might have influenced the results. Third, although our students have no standardized EFAST training and assessment in the early years of medical school, it is possible that some of the students might have observed EFAST or other ultrasound applications during their other rotations. Fourth, our sample size is small which may not be enough to prove that 1-hour didactic teaching and 4-hours practical education are enough for EFAST training. Fifth, we have used normal human models to train our students. There are other different training models with pathological findings like animal models which we did not use.²⁴ Sixth, the total number of EFAST exams in real patients and their severity may vary among students which can be a factor affecting the results.

6. Conclusions

We found that final year medical students could effectively learn EFAST examination. Five hours EFAST training and practice provide an acceptable level of skill for medical students. The marks of EFAST station in OSCE showed significant weak correlation with the other OSCE stations marks and the final clerkship marks. The EFAST OSCE station mark had no correlation with the final MCQ exam mark which assesses a different cognitive learning domain.

Ethics approval and consent to participate

This study was reviewed and approved by The Research and Graduate Studies Ethics Committee (Reference No: ERS-2016-4432). The study is a retrospective analysis of an assessment which does not include patient information and pictures, therefore consent to participate is not applicable to our study.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of interest

The authors declare that they have no competing interests. None declared.

Funding

No funding was used for the study

Authors' contributions

AAC, FMAZ, and MEZ conceived the study, designed the trial. AAC and AN supervised the conduct of the trial and data collection. FMAZ provided statistical advice on study design and analyzed the data. AAC and FMAZ drafted the manuscript, and all authors contributed substantially to its revision. AAC takes responsibility for the paper as a whole.

Abbreviations

EFAST

Extended Focused Assessment Sonography for Trauma

EMC

Emergency Medicine Clerkship

MCQ

Multiple Choice Question

OSCE

Objective Structured Clinical Examination