Objective Structured Clinical Examination (OSCE)-based Assessment of the Advanced Trauma Life Support (ATLS) Course in Iran

Abstract

Objective:

To evaluate the effect of advance trauma life support (ATLS®) training on general surgery residents clinical reasoning skills using the national boards-style objective structured clinical examination (OSCE).

Methods:

This cross-sectional single-center study was conducted in Shiraz University of Medical Sciences including 51 surgery residents that participated in a mandatory national board style OSCE between May 2014 and May 2015. OSCE scores of two groups of general surgery residents including 23 ATLS® trained and 28 non-ATLS® trained were compared using Mann-Whitney U test. The exam was graded out of 20 points and the passing score was ≥14 including 40% trauma cases.

Results:

There were 8(15.7%) women and 43(84.3%) men among the participants with mean age of 31.12 ± 2.69 and 33.67 ± 4.39 years in women and men respectively. Overall 7 (87.5%) women and 34 (79.07%) men passed the OSCE. The trauma section OSCE score was significantly higher in the ATLS® trained participants when compared to non-ATLS®(7.79 ± 0.81vs.6.90 ± 1.00; p=0.001). In addition, the total score was also significantly higher in ATLS® trained residents (16.07 ± 1.41 vs. 14.60 ± 1.40; p=0.001). There was no association between gender and ATLS® score (p=0.245) or passing the OSCE (p=0.503).

Conclusion:

ATLS® training is associated with improved overall OSCE scores of general surgery residents completing the board examinations suggesting a positive transfer of ATLS learned skills to management of simulated surgical patients including trauma cases.

Introduction

Trauma is a leading cause of mortality in the first four decades of life. Training of physicians in the management of the acutely injured patients should therefore be in a high priority of medical education [1, 2]. The Advanced Trauma Life Support (ATLS®) course was established in 1978 and adopted by the American College of Surgeons in 1980 as the primary course for training physicians in trauma resuscitation [3]. Since  its inception over three decades ago the ATLS® course has changed in-hospital management of major trauma patients and is now accepted as a standard of care in more than 60 countries worldwide with over a million physicians trained [1, 4]. In Iran the ATLS® program was introduced in 2012 by Atieh Hospital. Shahid Rajaee Trauma Center was the second center with nine courses having been conducted by Trauma Research Center, Shiraz University of Medical Science, Shiraz, Iran.As an intensive and highly standardized method of training, there has been extensive interest in quantification of the overall clinical impact of the course. The impact has been measured in terms of staff knowledge and patient outcome. In multiple studies, performed in developing and developed countries, the course has been reported to improve knowledge and attitude of participants with various levels of experience [5-7].  A positive impact on the outcome of trauma patients has also been suggested as a result of ATLS® training of the emergency room staff [8,9]. While the positive effect on staff knowledge is less controversial, the effect on patient outcome is yet to be proved in properly performed RCT’s [4, 10]. Following accreditation of the first centers in Iran, general surgery residents have been randomly exposed to the course. This has provided a unique opportunity to evaluate the effect of ATLS® training on the performance of general surgery residents by comparing those with and without ATLS® training. To our knowledge this is the first study to evaluate the impact of the course on overall clinical reasoning and decision making skills of general surgery residents in an OSCE setting. The purpose of this study, therefore, is to assess the effect of ATLS® training on general surgery residents’ performance in the mandatory OSCE designed for general surgery board certification.

Materials and Methods

Study populationThis was a cross-sectional single-center study, including a group of 51 general surgery residents who had participated in a mandatory national board style OSCE in Shiraz University of Medical Sciences between May 2014 and May 2015. The backgrounds of these surgery residents were similar in that they all took their undergraduate training at the Shiraz University of Medical Sciences and were practicing in the same  major hospitals that managed similar numbers of blunt and penetrating injuries. The study protocol was approved by the institutional review board and the medical ethics committee of Shiraz University of Medical Sciences. All the participants provided their informed written consents to use their educational background information for the study. 

ATLS® courseThe ATLS® course is conducted with approval of the American College of Surgeons. Before the course, the surgery residents receive the course manual. During a 2-day course with an instructor-to candidate ratio of 1:4, residents are trained by instructors. These instructors are mostly surgeons and anesthesiologists but also included physicians from other fields. During the course, basic emergency measures are taught and reviewed. The objective of the course is to enable participants to independently manage trauma patients using the correct order of priority. The objectives are reached using interactive lectures, practical skill stations, group discussions, and simulated clinical scenarios. During the study period, the course was taught from the 9th edition of the ATLS® Student Course Manual [11]. Assessment of candidates included a multiple choice question (MCQ) examination supplied by the American College of Surgeons Committee on trauma, in which the participant needs to achieve a mark of 80 % to pass. During the course, attention is also given to the multidisciplinary character of trauma care and the organization and logistics of trauma care in hospitals and the surrounding area. (These few provider courses were held in Farsi as sanctioned by the ACS, since no international participant was attending.)

Study protocol We recorded the baseline characteristics of all the participants including age, gender, undergraduate medical school and the medical training history. We also recorded the OSCE score of all the participants. Clinical skills were assessed utilizing the OSCE of National Surgical Boards. The overall score for OSCE was standardized to a maximum score of 20. A passing score was considered to be 14/20 or 70%.

Statistical analysisAll statistical analyses were performed using statistical package for social sciences (SPSS Inc., Chicago, Illinois, USA) version 11.0. Data are presented as mean ± SD and proportions as appropriate. Independent t-test was used to compare the results of the OSCE between study groups. Variables without normal distribution were compared using Mann-Whitney U-test.Chi square test was used to compare the proportions between two study groups. A two-tailed p-value of less than 0.05 was considered statistically significant.

Results

Overall we included a total number of 51 general surgery residents who participated in the study among whom there were 23 (45.1%) ATLS® trained and 28 (54.9%) non-ATLS® trained residents. Among the participants there were 43 (84.3%) men and 8 (15.7%) women with mean age of 33.67±4.39and 31.12±2.69 years, respectively. There was no significant difference between two study groups regarding the age (p=0.245) and gender (p=0.503). Out of the 51 residents, 41(80.39%) passed the OSCE and 10 (19.61%) did not.The main characteristics of participants and the OSCE score in ATLS® and non-ATLS® trained groups are summarized in Table 1.We found that the mean OSCE score in ATLS® trained group was significantly higher when compared to non-ATLS® trained group (16.07±1.41vs. 14.60±1.40; p=0.001), respectively. We also found that the rate of passing of OSCE examination was significantly higher in ATLS® trained compared to non-ATLS® trained group (p=0.021). The trauma section OSCE score was also significantly higher in the ATLS® trained participants when compared to non-ATLS® residents (7.79±0.81 vs. 6.90±1.00; p=0.001). There was no association between gender and ATLS® score (p=0.245) or passing the OSCE (p=0.503).Table 2 summarizes the characteristics of the participants according to the OSCE results. The trauma score in OSCE was significantly higher in those who passed OSCE when compared to those who did not (8.3±1.6 vs. 5.6±2.6; p<0.001).

Table 1.

The characteristics and OSCE scores of 51 general surgery residents who had attended the ATLS® course (n=23) or did not (n=28).

	ATLS® a trained (n=23)	Non-ATLS®trained (n=28)	p -value
Age (years)	34.4 ± 4.5	32.3 ± 3.8	0.089
Gender
Men (%)	18 (78.3%)	25 (89.3%)	0.442
Women (%)	5 (21.7%)	3 (10.7%)
OSCE b score	16.04 ± 1.41	14.61 ± 1.40	0.001
Passed (%)	21 (91.3%)	20 (71.4%)	0.021
Failed (%)	2 (8.7%)	8 (28.6%)
Trauma section score	7.79 ± 0.81	6.90 ± 1.00	0.001

^a ATLS®: Advanced Trauma Life Support;

^b OSCE: Objective Structural Clinical Examination

Table 2.

The characteristics of 51 general surgery residents according to OSCE results

	OSCE passed (n=41)	OSCE b failed (n=10)	p -value
Age (years)	32.6 ± 4.2	33.4 ± 4.3	0.581
Gender
Men (%)	34 (82.9%)	9 (90.0%)	0.503
Women (%)	7 (17.1%)	1 (10.0%)
Trauma section score	8.3 ± 1.6	5.6 ± 2.6	<0.001
ATLS® a course
Trained (%)	21 (51.2%)	2 (20.0%)	0.021
Non-trained (%)	20 (48.8%)	8 (80.0%)

^a ATLS®: Advanced Trauma Life Support;

^b OSCE: Objective Structural Clinical Examination

Discussion

This study evaluated the relationship between ATLS® training and success in OSCE of National Surgical Boards in surgery residents of Shiraz University of Medical Sciences, Shiraz, Iran. We found that residents that have attended the ATLS® training course, obtained significantly higher scores in the national boards-style OSCE. ATLS® teaches a systematic approach for the assessment of trauma patients, originally designed for emergency situations in rural settings where as few as one doctor and one nurse may be available at any one time. The course is accepted as a standard of care for the first hour in trauma centers around the world [12]. As a course aiming at improving outcome of trauma patients, there have been various studies evaluating effectiveness of the ATLS course on patient outcome and staff competency, as summarized in a recent review [10].Knowledge and competency of ATLS trained physicians and medical students have been reported in various settings with fairly uniform results [11, 13]. It has been reported that doctors who deal clinically with all aspects of airway, breathing, and circulation of the ATLS® course are more likely to pass the MCQ exam in UAE [1]. Similarly in a group of practicing physicians Ali et al. found effectiveness of the ATLS® program among practicing physicians as measured by improvement in OSCE scores, adherence to trauma priorities, maintenance of an organized approach to trauma care, and cognitive performance in MCQ examinations [9]. In this study we compared overall performance of ATLS® trained and non-ATLS trained general surgery residents in a standardized OSCE which contains 40% trauma cases. The organized insight the course provides in trauma management as well as the overall educational impact of the course both might have resulted in the improved performance. Similar to the previous studies we found ATLS® trained physicians outperforming their non-ATLS® trained colleagues, as demonstrated  among general surgery residents with 3 years dedicated training  in surgery and trauma. In terms of actual reduction of trauma patient mortality and morbidity however, results are not uniform.  In the UK, the implementation of ATLS® improved trauma care and reduced the number of preventable deaths of trauma patients [13]. Similarly the effectiveness of the ATLS® program was evaluated by Ali et at utilizing OSCE methodology in a group of practicing physicians who had applied for an ATLS® program in Trinidad and Tobago [9]. Another study revealed a significant improvement in outcome of trauma patients. The considerable positive impact of the ATLS® program in Trinidad and Tobago has also been assessed by studying its influence on the institution of resuscitative techniques [14].Contrary to these results, however, in a study performed in Greece evaluating the impact of ATLS® on trauma mortality in a non-trauma system setting  ATLS®  trained  physicians reportedly achieved worse outcomes than their non- trained colleagues when managing trauma patients [4]  This is however a large community-based retrospective study in which ATLS® trained physicians were not alone in managing the patients , so the outcome could not be ascribed entirely to the ATLS® training. Moreover, in this study was not only higher in ATLS® trained physicians, but also for all specialty-trained physicians.  Our study evaluated impact of the course on OSCE performance and cannot be directly compared to these studies. When translated into patient outcome it may be argued that many uncontrolled factors unrelated to physician training may affect the results in clinical situations. The OSCE, used as the main tool for assessment of resident competency in this study, was devised to assess clinical skills by utilizing standardized simulated patients in 1975 [6]. In addition to clinical examination, it evaluates competence in skills such as communication, medical procedures and interpretation of results [6]. Although, OSCE has been characterized as an expensive, time consuming, and, in certain cases, stressful method of evaluation, it is less potentially biased, especially in terms of examiner partiality. In addition, it has a high level of reliability and validity, which depends on elements such as: scenario and variety of contents, checklist and scoring methods, number of stations and examiners per station and simulated patients [8]. This technique provides an objective assessment of clinical performance and has been regarded as the most effective tool for assessing clinical competence of surgical residents. The only other study using the OSCE for evaluation of ATLS® outcome that we are aware of is performed on practicing physicians without specialty surgery training [9]. Therefore we believe using the highly standardized surgical board’s style OSCE in surgery residents utilized in this study provides objective evidence of the educational value of the course in this group of physicians.There have been concerns over the value of ATLS® in non-English speaking countries [4, 8, 9, 15, 16]. The ATLS® has allowed using local languages in courses with no international attendance. Therefore, in this study, possible confounding effect of non-uniform English competency was controlled for by holding the courses in Farsi which was the native language of the participants.We believe that our data is of value in that it compares ATLS® and non-ATLS® trained surgery residents’ performance in the same standardized exam (National Surgical Boards style OSCE) and in the same time period. As far as we know educational value of the course has not been evaluated in general surgery residents before. However, this study is limited by a small sample size and short duration of study. Also final result of the training in the community in terms of patient outcome was not evaluated.In conclusion, OSCE scores in National Surgical Boards exams among ATLS® trained residents were higher than those among non ATLS® trained residents. Improved success in this highly standardized test, used for licensing future surgeons can be attributed to the in-depth insight the course provides in trauma management and possibly also in clinical reasoning in the emergency setting in general.

Conflict of interest

The authors declare no conflict of interest for this work.