Abstract
BACKGROUND:
In this study, we attempted to assess the change of perception of postgraduate students on objective structured clinical examination (OSCE) in burns and plastic surgery after first five OSCE.
METHODS:
A prevalidated feedback questionnaire was used to assess and score the perception of postgraduate students on OSCE in burns and plastic surgery. The opinion of postgraduate students on the feedback questionnaire after first and fifth assessment tests based on OSCE was analyzed. The results were compiled on a data sheet and analyzed using Microsoft Excel and plotted as graphical interpretation. The statistical analysis was done using MedCalc software.
RESULTS:
The results of the study showed that there is a positive change in perception of students in favor of monthly assessment based on OSCE in burns and plastic surgery after fifth assessment. The mean students’ favorable perception score after the first assessment with was 30.2 ± 2.828 (mean ± standard deviation [SD]) and after fifth assessment, 43 ± 2.828 (mean SD) with Student's test t = 10.119 and P < 0.0001 which is highly significant statistically in favor of OSCE after fifth assessment.
CONCLUSIONS:
We observed in our study that the monthly assessment based on OSCE is well accepted by the students of our subspecialty after few assessments; however, further studies are required to augment the evidence.
Keywords: Clinical examination, examination assessment tool, objective structured
Introduction
The objective structured clinical examination (OSCE) is being conducted monthly in our institution for student skill assessment in all departments. Periodic assessment of postgraduate students of Magister Chirurgiae (M Ch) burns and plastic surgery postgraduate training program of most medical schools based on different methods is a standard norm. These assessments are important in helping a student to become aware about his shortcomings and hence plan a course correction before appearing for the final assessment. It also helps the teachers in medical education to evaluate the level of interest and appreciation of a subject. The OSCE is interesting tool which we are applying at our institution to test the cumulative knowledge and skills acquired by a student. These assessment tools are to be evaluated for the objectives these are intended for. The prevalidated feedback questionnaire is an essential component of the evaluation system tool to complete the loop to assess the efficacy of that tool of assessment.
Materials and Methods
Study design and setting
The prospective comparative study on the perception of postgraduate students on OSCE was conducted in the postdoctoral department of burns and plastic surgery of our institution from February 2019 to April 2021.
Study participants and sampling
The theoretical knowledge and its application in clinical and practical skills of all ten students of our department were assessed by OSCE stations.
Data collection tool and technique
The OSCE stations included a total of four stations with predefined scenario-one each for soft skills, clinical case and planning, clinical radiology, instruments, and pathological specimen. Each station had a structured question which had to be answered by performing specific skills. The observer had to score the students according to the predefined checklist. Immediately after the examination, a prevalidated feedback questionnaire [Table 1] was used to assess the students’ favorable perception score (SFPS) for OSCE in burns and plastic surgery. The written consent from each participant was taken, and the identity of participants has not been revealed at any stage of the study conducted in accordance with the declaration of Helsinki version 2013. The questionnaire used was validated by three independent researchers. The prevalidated questionnaire [Table 1] comprises 15 questions with three response options which are agree, neither agree nor disagree, and disagree. The response agrees, neither agree nor disagree, and disagree carries score of 3, 2, and 1, respectively. The total score from each participant called SFPS by us is calculated accordingly by adding the score against each question. The SFPS for OSCE after first [Table 2] and fifth [Table 3] assessments were compiled on a data sheet and analyzed using Microsoft Excel and plotted as graphical interpretation.
Table 1.
Questionnaire for survey on the perception of postgraduate students after first and fifth objective structured clinical examination in burns and plastic surgery
| Name (optional): Age: Years: Sex: Male/female | |||
| Please tick the response: After first OSCE: ( ), After fifth OSCE: ( ) | |||
|
| |||
|---|---|---|---|
| Question | Agree score 3 | Neither agree Nor disagree Score 2 | Disagree Score 1 |
| 1. Examination conducted was fair? | ( ) | ( ) | ( ) |
| 2. Examination was not stressful? | ( ) | ( ) | ( ) |
| 3. Examination was not difficult? | ( ) | ( ) | ( ) |
| 4. Examination was well structured? | ( ) | ( ) | ( ) |
| 5. Wide knowledge area covered? | ( ) | ( ) | ( ) |
| 6. Questions and procedures asked were fair? | ( ) | ( ) | ( ) |
| 7. Students aware of level of information required? | ( ) | ( ) | ( ) |
| 8. Examination provided opportunity to learn? | ( ) | ( ) | ( ) |
| 9. Examination helped to highlight your weaknesses? | ( ) | ( ) | ( ) |
| 10. Time for examination was sufficient? | ( ) | ( ) | ( ) |
| 11. Score was standardized? | ( ) | ( ) | ( ) |
| 12. Examination pattern is true measure of clinical and soft skills? | ( ) | ( ) | ( ) |
| 13. Attitude of examiner is better? | ( ) | ( ) | ( ) |
| 14. Examination pattern may influence method of teaching? | ( ) | ( ) | ( ) |
| 15. Examination pattern should be used as method of assessment in burns and plastic surgery? | ( ) | ( ) | ( ) |
OSCE=Objective structured clinical examination
Table 2.
Students’ favorable perception score after first objective structured clinical examination in burns and plastic surgery
| Student number | Agree (score) | Neither agree nor disagree (score) | Disagree (score) | SFPS |
|---|---|---|---|---|
| 1 | 12 | 16 | 3 | 31 |
| 2 | 9 | 18 | 3 | 30 |
| 3 | 15 | 14 | 3 | 32 |
| 4 | 12 | 14 | 4 | 30 |
| 5 | 6 | 18 | 4 | 28 |
| 6 | 12 | 16 | 3 | 31 |
| 7 | 9 | 18 | 3 | 30 |
| 8 | 15 | 14 | 3 | 32 |
| 9 | 12 | 14 | 4 | 30 |
| 10 | 6 | 18 | 4 | 28 |
SFPS=Students’ favorable perception score
Table 3.
Students’ favorable perception score after fifth objective structured clinical examination in burns and plastic surgery
| Student number | Agree (score) | Neither agree nor disagree (score) | Disagree (score) | SFPS |
|---|---|---|---|---|
| 1 | 42 | 2 | 0 | 44 |
| 2 | 39 | 2 | 1 | 42 |
| 3 | 42 | 2 | 0 | 44 |
| 4 | 42 | 0 | 1 | 43 |
| 5 | 39 | 2 | 1 | 42 |
| 6 | 42 | 2 | 0 | 44 |
| 7 | 39 | 2 | 1 | 42 |
| 8 | 45 | 0 | 0 | 45 |
| 9 | 42 | 0 | 1 | 43 |
| 10 | 39 | 0 | 2 | 41 |
SFPS=Students’ favorable perception score
Statistical analysis
The statistical analysis was done using MedCalc statistical software. MedCalc Software Ltd Acacialaan 22 8400 Ostend Belgium. The paired t-test was used to compare the quantitative data after the first and fifth assessment tests. Statistical significance is set at 5% (P < 0.05).
Ethical consideration
The study (protocol/02/BPS) is done in accordance with declaration of Helsinki version 2013, and the identification of participants is not disclosed.
Results
The opinion of postgraduate students recorded as per the feedback questionnaire [Table 1] after first and fifth assessment tests were recorded, and statistical results were tabulated in Tables 2 and 3, respectively, with subsequent statistical comparison of the mean SFPS after first and fifth assessment [Table 4] using OSCE in burns and plastic surgery. The mean SFPS after the first assessment with was 30.2 ± 2.828 (mean ± standard deviation [SD]) and after fifth assessment 43 ± 2.828 (mean SD) with Student's test t = 10.119 and P < 0.0001 which is highly significant statistically in favor of OSCE after fifth assessment. This shows that there is a positive change in perception of students in favor of assessment based on OSCE in burns and plastic surgery [Figure 1].
Table 4.
Comparison of the perception of postgraduate students after first and fifth objective structured clinical examination in burns and plastic surgery
| OSCE | Mean SFPS±SD | Student’s t-test | Level of significance (P) |
|---|---|---|---|
| After first OSCE | 30.2±2.828 | 10.119 | <0.0001 |
| After fifth OSCE | 43±2.828 |
OSCE=Objective structured clinical examination, SFPS=Students’ favorable perception score, SD=Standard deviation
Figure 1.
Line chart of students’ favorable perception score after first and fifth objective structured clinical examination
Discussion
The key components of the learning climate in the medical education system are teacher, learner (student), curriculum, assessment, and evaluation. The evaluation of the learning climate and assessment tool is performed through a feedback system from the learner and teacher. The Accreditation Council for Graduate Medical Education classifies medical competence into six domains: medical knowledge, patient care, professionalism, interpersonal and communication skills, systems-based practice, practice-based learning, and improvement.[1] The clinical vignette-based multiple-choice question paper is considered to be a valid tool for assessing the medical knowledge and its application in decision-making and planning while OSCE for other five domains.[1,2] Thus, OSCE is considered the gold standard of assessment methods.[3] However, OSCE has two major drawbacks that it is expensive and time-consuming.[4] There are number of studies published which have showed a positive change in perception of undergraduate students in favor of their assessment based on OSCE.[5,6,7,8,9] The recent study also showed that the Student's perception was positive especially regarding to organization and the time attributed to each station, and furthermore, the students considered that the topics and questions applied in each station were relevant.[10] Our study has showed a positive change [Figure 1] in perception of students in favor of assessment of students in burns, plastic, and reconstructive surgical super specialty based on OSCE. Since the significance of student feedback for assessment tool in undergraduate and postgraduate medical education is being increasingly recognized,[11] we conducted such study on perception of students of plastic surgery super specialty on objective structured clinical/practical examination.
Our students were generally satisfied with assessment system as indicated by their positive feedback and high mean students’ favorable score after few assessments using OSCE. However, our study was limited by sample size which can be increased in future studies and may need to augment the evidence by conducting multi-institutional or multiple such studies.
Conclusions
We observed in our study that the monthly assessment based on OSCE is well accepted by the students of our subspecialty after few assessments, however, further studies are required to augment the evidence.
Ethical statement
The written consent from each participant was taken and the identity of participants has not been revealed at any stage of the study conducted in accordance with the declaration of Helsinki version 2013.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
References
- 1.Chan CY. Is OSCE valid for evaluation of the six ACGME general competencies? J Chin Med Assoc. 2011;74:193–4. doi: 10.1016/j.jcma.2011.03.012. [DOI] [PubMed] [Google Scholar]
- 2.Varkey P, Natt N, Lesnick T, Downing S, Yudkowsky R. Validity evidence for an OSCE to assess competency in systems-based practice and practice-based learning and improvement: A preliminary investigation. Acad Med. 2008;83:775–80. doi: 10.1097/ACM.0b013e31817ec873. [DOI] [PubMed] [Google Scholar]
- 3.Swing SR. Assessing the ACGME general competencies: General considerations and assessment methods. Acad Emerg Med. 2002;9:1278–88. doi: 10.1111/j.1553-2712.2002.tb01588.x. [DOI] [PubMed] [Google Scholar]
- 4.Barman A. Critiques on the objective structured clinical examination. Ann Acad Med Singap. 2005;34:478–82. [PubMed] [Google Scholar]
- 5.Al-Naami MY. Reliability, validity, and feasibility of the Objective Structured Clinical Examination in assessing clinical skills of final year surgical clerkship. Saudi Med J. 2008;29:1802–7. [PubMed] [Google Scholar]
- 6.Martin IG, Jolly B. Predictive validity and estimated cut score of an objective structured clinical examination (OSCE) used as an assessment of clinical skills at the end of the first clinical year. Med Educ. 2002;36:418–25. doi: 10.1046/j.1365-2923.2002.01207.x. [DOI] [PubMed] [Google Scholar]
- 7.Khosravi Khorashad A, Salari S, Baharvahdat H, Hejazi S, Lari SM, Salari M, et al. The assessment of undergraduate medical students’ satisfaction levels with the objective structured clinical examination. Iran Red Crescent Med J. 2014;16:e13088. doi: 10.5812/ircmj.13088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Raheel H, Naeem N. Assessing the Objective Structured Clinical Examination: Saudi family medicine undergraduate medical students’ perceptions of the tool. J Pak Med Assoc. 2013;63:1281–4. [PubMed] [Google Scholar]
- 9.Elfaki OA, Al-Humayed S. Medical students’ perception of OSCE at the department of internal medicine, college of medicine, King Khalid University, Abha, KSA. J Coll Physicians Surg Pak. 2016;26:158–9. [PubMed] [Google Scholar]
- 10.de Oliveira FA, Porto FR, Ribeiro CG, Haddad AE, de Oliveira RG, Ferraz Júnior AM. Objective structured clinical examination, OSCEs: An advance in the teaching and learning process in the student's perception. Rev Odontol UNESP. 2019;48:e20190027. [Google Scholar]
- 11.Al-Mously N, Nabil N, Salem R. Student feedback on OSPE: An experience of a new medical school in Saudi Arabia. J Int Assoc Med Sci Educ. 2012;22:10–6. [Google Scholar]