Abstract
BACKGROUND:
The mini-clinical evaluation exercise (mini-CEX) is a formative assessment tool designed to provide feedback on skills essential to good medical care by observing an actual clinical encounter. However, the bigger advantage of mini-CEX is the structured feedback that it provides to the students as well as the faculty, thus helping them to make better decisions.
MATERIALS AND METHODS:
This study was a cross-sectional observational study. Sixteen surgery residents volunteered for participation and five professors conducted sessions; hence, 80 mini-CEX encounters. Seven core clinical skill assessments were done, and the performance was rated on a 9-point scale (grouped into unsatisfactory, satisfactory, and superior). Immediate feedback to the residents was given by the faculty. Delayed feedback from faculty and residents regarding the perception of mini-CEX was taken. Statistical analysis was done using SPSS version 20 and analysis of variance (ANOVA) for inferential statistics.
RESULTS:
As planned, 80 (100%) mini-CEX encounters were conducted. Surgery residents showed improvement that was statistically significant in the competencies of medical interviewing skills, physical examination skills, humanistic qualities/professionalism, and counseling skills. Most of the faculty (80%) were able to identify the gaps in the knowledge of students and areas of improvement for their teaching. However, 60% of the faculty felt that it required more effort than traditional methods. The mean time taken by the assessor for observation and feedback to residents was 12.51 min and 5.68 min, respectively. The mean scores of evaluator satisfaction and resident satisfaction with mini-CEX sessions were 6.04 and 7.49, respectively.
CONCLUSIONS:
Mini-CEX improves the learning environment in residency and also leads to improvement in medical interviewing skills, physical examination skills, humanistic qualities/professionalism, and counseling skills. It is done in the actual patient encounter and hence prepares the resident better for dealing with patients in the future.
Keywords: Clinical competence, medical interviewing skills, professionalism, residents
Introduction
Despite an increasing emphasis on workplace-based assessment (WPBA) during medical training, the existing system largely relies on summative assessment, whereas formative assessment is less valued. Various tools have been described for workplace-based assessment, mini-clinical evaluation exercise (mini-CEX) being one of them. mini-CEX is well accepted in western countries; however, it is scarcely used in India.[1]
Mini-CEX is a useful method of assessing clinical competence. It is called mini because it takes comparatively less time than a conventional case presentation. However, the bigger advantage of mini-CEX is the structured feedback that it provides to the students as well as the faculty, thus helping them to make better decisions.[2] Mini-CEX is a 10 to 20 min snapshot of doctor/patient interaction. It is designed to assess the clinical skills, attitudes, and behaviors of students essential to providing high-quality care.
Students are asked to undertake four to six observed encounters with different observers for each encounter. Each of these encounters represents a different clinical problem, and trainees should sample from each of the core problem groups identified as important (for example history taking, physical examination, diagnosis, communication, counseling, etc.). However, not all elements need to be assessed at each encounter. Each encounter takes about 20 min, with the first 15 min for the encounter and the last 5 min for feedback. Immediate feedback is provided by the person assessing the performance.[2] Mini-CEX conforms to the highest level of Miller's pyramid, that is, “Does.”[3]
In a study by Norcini et al.[4] it has been concluded that even though the measurement characteristics of the mini-CEX are similar to those of performance assessments, which utilize standardized patients, the difficulty of the mini-CEX varies with the patients that a resident encounters. This effect can be mitigated to a degree by the examiners, who slightly overcompensate for patient difficulty, and by the fact that each resident interacts with several patients. Furthermore, the mini-CEX has higher fidelity than other formats, permits evaluation based on a much broader set of clinical settings and patient problems, and is administered on-site, in a real setting.
In our institution, surgery residents do not have a structured assessment program during residency, and the focus is on the summative assessment at the end of the course. Similarly, there is no structured feedback that can help the residents to take remedial actions well in time. Mini-CEX, a workplace-based assessment tool can prove beneficial in identifying the gaps in knowledge and any improvement required in teaching. Hence, this study was conducted for the application of mini-CEX for surgery residents and to obtain the perception of faculty as well as the residents toward mini-CEX.
Materials and Methods
Study design and setting
The project was conducted at a tertiary care institute having residents in the department of surgery from May 2019 to September 2019.
Study participants
There were seven professors in the Department of Surgery at the time the study was planned. Out of these, five professors gave their consent to be part of the study. They were sensitized before conducting mini-CEX sessions.
Data collection tool and technique
-
1)
The concept of mini-CEX was explained to the faculty members and the systematic ways and methods of giving feedback were conveyed to them.
-
2)
All 16 surgical residents who consented were sensitized toward mini-CEX.
-
3)
Preparation of a feedback questionnaire for students and faculty was done and the questionnaire was validated by the peer group.
-
4)
Five sessions of mini-CEX were conducted for each resident in different settings of the out-patient department (OPD), in-patient department (IPD), and emergency ward with an emphasis on not repeating the clinical cases. It was taken care that each student should have at least one session in each setting, that is, OPD, IPD, and emergency.
-
5)
The students were assessed using standard mini-CEX Proforma testing 7 core clinical skills rated on a 9-point scale. A score of 1–3 was considered unsatisfactory, a score of 4–6 was considered satisfactory, and a score of 7–9 was considered superior [Figure 1].
-
6)
Immediate feedback was provided to the students by the faculty.
-
7)
Duration of each mini-CEX was recorded.
-
8)
The difficulties were identified, recorded, and appropriate measures were taken to address them.
-
9)
At the conclusion, the opinion of residents and faculty members regarding their experience with mini-CEX was taken using a pre-validated questionnaire. The results were analyzed using statistical tools (Ethical code no. – IEC-138).
Figure 1.
Mini-CEX proforma
Statistical analysis of all components of mini-CEX
Analysis was done by SPSS version 20 to calculate the descriptive statistics
P-value of < 0.05 was taken to be significant
Analysis of variance (ANOVA) was used to calculate inferential statistics.
Ethical consideration
The study was conducted only after approval from the institutional ethics committee (IEC-138) and after obtaining due consent from participating faculty and surgery residents.
Results
As planned, 80 (100%) mini-CEX encounters were conducted. Five professors of surgery conducted one session each with all 16 surgery residents.
Patient setting
Forty sessions (50%) were conducted with ambulatory patients, 24 (30%) sessions were for inpatients, and 16 (20%) were conducted in the emergency department.
Focus areas
The focus areas during the evaluation were (according to the evaluating professor)
Data gathering 91%
Diagnosis 51%
Therapy 34%
Counseling 59%.
Complexity of cases
As far as the complexity of cases was concerned, 32 (40%) were in the low category, 26 (32.5%) were in the moderate category, and 22 (27.5%) were in the high category.
Competencies of residents assessed on a 9 point scale
A mini clinical evaluation exercise was performed, and students were marked on a 9-point scale. The marks from 1 to 3 were considered in the group unsatisfactory, a score of 4 to 6 was included in the satisfactory group, and a 7 to 9 score was considered superior. The change in the number of students in all categories for each competency from the initial to the final session is depicted in Table 1.
Table 1.
Showing change in the number of students in all categories
| Session | Unsatisfactory | Satisfactory | Superior | |
|---|---|---|---|---|
| Medical interviewing skillsw | Initial session | 5 (31.25%) | 11 (68.75%) | 0 |
| Final session | 2 (12.5%) | 13 (81.25%) | 1 (6.25%) | |
| Physical examination skills | Initial Session | 7 (43.75%) | 9 (56.25%) | 0 |
| Final Session | 1 (6.25%) | 14 (87.5%) | 1 (6.25%) | |
| Humanistic qualities/professionalism | Initial Session | 7 (43.75%) | 9 (56.25%) | 0 |
| Final Session | 2 (12.5%) | 11 (68.75%) | 3 (18.75%) | |
| Clinical judgment | Initial Session | 3 (18.75%) | 13 (81.25%) | 0 |
| Final Session | 2 (12.5%) | 13 (81.25%) | 1 (6.25%) | |
| Counseling skills | Initial Session | 8 (50%) | 8 (50%) | 0 |
| Final Session | 2 (12.5%) | 10 (62.5%) | 4 (25%) | |
| Organization efficiency | Initial Session | 4 (25%) | 12 (75%) | 0 |
| Final session | 3 (18.75%) | 12 (75%) | 1 (6.25%) | |
| Overall clinical competence | Initial Session | 1 (6.25%) | 15 (93.75%) | 0 |
| Final Session | 1 (6.25%) | 13 (81.25%) | 2 (12.5%) |
On application of ANOVA, there was a statistically significant difference in scores of medical interviewing skills, physical examination skills, professionalism, and counseling skills.
On multiple comparisons, the following differences were seen:
-
1)
Medical interviewing skills: The statistically significant improvement in scores was seen from the first to fourth sessions.
-
2)
Physical examination skills: The statistically significant improvement in scores was seen from the first to fourth sessions.
-
3)
Humanistic qualities/professionalism: The statistically significant improvement in scores from the first session was seen only by the time the fifth session was conducted.
-
4)
Clinical judgment: There was no statistically significant improvement seen in scores of clinical judgments.
-
5)
Counseling skills: In the counseling skill competency, the statistically significant difference of scores was found from second to fifth sessions.
-
6)
Organization efficiency: There was no statistically significant improvement seen in scores of clinical judgments.
-
7)
Overall clinical competence: This competency of overall clinical competence also did not show statistically significant improvement in scores over the course of sessions.
The median score in different skills of residents is shown in Table 2.
Table 2.
Showing median score in different skills of residents
| Competency | Median score |
|---|---|
| Medical interviewing skills | 5 |
| Physical examination skills | 4 |
| Humanistic qualities/professionalism | 5 |
| Clinical judgment | 5 |
| Counseling skills | 5 |
| Organization efficiency | 4 |
| Overall clinical competence | 5 |
The mean time taken by the assessor for observation was 12.51 min (7 min–20 min).
The mean time taken for feedback to the resident by the assessor was 5.68 min (4 min–10 min).
The mean score of evaluator satisfaction was 6.04 (4–9). The mean score of resident satisfaction with mini-CEX sessions was 7.49 (4–9) and showed that residents had good-to-high satisfaction rates with the exercises conducted.
After the completion of the sessions, both faculty and residents were requested to fill out the feedback questionnaire. Faculty feedback and residents’ feedback are shown in Figures 2 and 3, respectively.
Figure 2.
Showing feedback from faculty where. (a) Any improvement in clinical skills was noted. (b) Can prove helpful to improve clinical and communication skills. (c) Valid method for formative assessment. (d) Can become a good formative assessment tool. (e) Adequate time devoted. (f) More effort than traditional methods. (g) Improvement in learning attitude of residents. (h) Able to identify gaps in knowledge and areas of improvement in their teaching methods
Figure 3.
Resident's feedback (a) Valid method for formative assessment. (b) Recommendation for the continuation of activity. (c) Effective formative assessment tool. (d) Sufficient time for the complete session. (e) Adequate time is given for feedback. (f) Anxious/uncomfortable during the session due to observation. (g) Feedback from teachers motivated to improve. (h) More comfortable in communication with patients and relatives. (i) Improvement in clinical skills. (j) Non/less-threatening environment. (k) Able to identify gaps in knowledge
Discussion
We observed that conducting mini-CEX for surgery residents in the Indian setting is feasible without utilizing too many resources, which has also been observed by other authors.[1,5,6]
We observed that 50% of our sessions were conducted with ambulatory patients, 43.75% of sessions were for inpatients, and 6.25% were conducted in the emergency department.
These results compared favorably with those of Norcini et al.,[7] in which 38% of sessions were conducted in ambulatory patients, 54% in IPD, and 14% in the emergency department (information on setting was missing for 4%). In yet another study conducted by Singh and Sharma,[5] 75% of sessions were conducted in the ambulatory setting (OPD) and 25% of sessions were conducted in the IPD setting.
The focus areas during the evaluation in our study were data gathering (91%), diagnosis (51%), therapy (34%), and counseling (59%). Similar results were reported by Goel and Singh,[8] where data gathering was 92%, diagnosis 60%, therapy 45%, and counseling was 30%. Singh and Sharma[5] reported data gathering to be 89.6%, diagnosis to be 62.7%, therapy to be 28.4%, and counseling to be 36.6%. While comparing the complexity of cases, we had 61.25% cases in the low category, 47.5% in the moderate category, and 3.75% were in the high category, whereas Singh and Sharma[5] reported 51.5% in the low category, 44% in the moderate category, and 4.5% were in the high category. In yet another study done by Liao et al.,[9] the observations were quite different. They reported 2.4% cases in the low category, 78.7% in the moderate category, and 10.7% were in the high category, whereas the data was missing in 8.2%.
In our study, the mean score was 5 for medical interviewing skills, humanistic qualities/professionalism, clinical judgment, and counseling, and overall clinical competency. The mean score was 4 for organizational efficiency and physical examination skills. Norcini et al.[7] reported a mean score of 6.5 for medical interviewing skills, physical examination, and clinical judgment; 7 for humanistic qualities, and 6.5 for overall competence. Chang et al.[10] reported a mean score of 6.6 for medical interviewing skills; 6.5 for physical examination, clinical skills, counseling skills, and organization/efficiency; 6.7 for clinical judgment and 6.8 for professionalism. The comparison of the median score of the present study with previous studies is shown in Table 3.
Table 3.
Showing comparison of median score of the present study with previous studies
| Competency | Median score | |
|---|---|---|
|
| ||
| Our study | Holmboe et al.[12] | |
| Medical interviewing skills | 5 | 6 |
| Physical examination skills | 4 | 6 |
| Humanistic qualities/professionalism | 5 | – |
| Clinical judgment | 5 | – |
| Counseling skills | 5 | 6 |
| Organization efficiency | 4 | – |
| Overall clinical competence | 5 | 6 |
We utilized a standard proforma for mini-CEX for the assessment of surgery residents. However, valid and reliable questionnaires can be developed for medical specialties as shown by Ghadrdoost et al.[11]
We observed that the mean time taken by the assessor for observation was 12.51 min, whereas the mean time taken for feedback to the resident by the assessor was 5.68 min.
Liao et al.[9] observed that the mean time taken by the assessor for observation was 19.18 min, whereas the mean time taken for feedback to the resident by the assessor was 14.99 min. More or less similar results were reported by Chang et al.[10] who observed assessor observation time to be 14.8 min and a feedback time of 11 min. Our sessions were quicker as compared to both of these authors.
The mean score of evaluator satisfaction was 6.04 in our study, whereas it was 7.98 in the study done by Liao et al.[9] and 6 in a Yale Primary Program care reported by Holmboe et al.[12]
The mean score of resident satisfaction with mini-CEX sessions was 7.49 in our study, whereas it was 7.96 as reported by Liao et al.[9] and 6 as reported by Holmboe et al.[12] Our study compared favorably resident feedback with the observations of Norcini et al.[7] and Burch et al.[13] In another study, the experiences of the participants showed that direct participation in the activities and healthcare techniques led to more sustainable learning for them.[14] In our study, 43.75% of residents reported that they felt anxious during the encounter, which was similar to the results reported by Malhotra et al.[15] Most of the faculty in our study felt that mini-CEX will prove to help improve the clinical and communication skills of residents and will lead to improvement in the learning attitude of the residents. Such sentiments were also echoed by Joshi et al.[1] Another study found that feedback content could make mini-CEX a rich evaluation instrument, and it was of great value in terms of critical and supportive feedback.[16]
Similarly, the positive responses received from both students and faculty highlight that the students are receptive to feedback, provided it is structured, constructive, and helps them to achieve their learning goals.[17]
Thus, mini-CEX facilitates feedback and can lead to improved trainee performance but the reluctance of assessors to award failing grades could potentially limit its ability to identify underperforming trainees.[18] This was also the problem faced in our study, where, in the initial encounters, the faculty was showing an affinity to give satisfactory scores but after repeated sensitization, they agreed to evaluate impartially and the sessions were conducted from the beginning after complete satisfaction of compliance.
The strength of the study lies in the fact that mini-CEX can be used as a tool for assessment as well as teaching clinical skills as a part of the workplace-based assessment. It can be used as a teaching method because it involves immediate feedback by the assessor, which can lead to improvement in various skills of the resident in further clinical encounters. Because it has a well-marked proforma covering all the aspects of the clinical encounter, it can be used as a clinical tool.
Limitation and recommendation
There are certain limitations of this study such as fewer residents, time constraints (small duration of the study), and non-consideration of the relationship between the complexity of cases and scoring. Despite these limitations, the application of mini-CEX helped the residents to identify the gaps in their knowledge. Most teachers were also able to identify the gaps in the knowledge of students and areas for improvement in their teaching. Students were objectively and regularly assessed. Students got motivated to modify their learning behavior. It is hoped that the continuation of this activity will help in achieving other intermediate outcomes of motivating teachers to modify their teaching methods.
The results have already shown improvement in the counseling skills of residents and hence it is expected that long-term outcomes of residents will become more competent clinically. Better doctor–patient communication and counseling skills can definitely be achieved with the continuation of mini-CEX sessions.
The results of feedback from faculty and residents were overwhelming. The faculties from other departments also appreciated our efforts and the findings of this study. We are planning to implement it for the subsequent batch of surgical residents. The findings have been appreciated by peers too during the dissemination of data. We are planning to recommend it to the National Medical Commission, India, to incorporate it as part of the surgery curriculum.
Conclusions
Mini-CEX is an acceptable and practical tool for the assessment of residents; especially formative assessment. It improves the learning environment in residency and leads to improvement in medical interviewing skills, physical examination skills, humanistic qualities/professionalism, and counseling skills. It was done in the actual patient encounter; hence, it is expected to prepare the residents to deal with patients in a better way in the future. However, a higher percentage of faculty felt that mini-CEX requires more effort than traditional methods.
Declaration of patient consent
The authors certify that they have obtained all appropriate patient consent forms. In the form, the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
Acknowledgments
We would like to acknowledge the valuable suggestions given by Dr. Dinesh K. Badyal (Convener-NMC, Nodal Centre Professor of Pharmacology and Dept. of Medical Education, CMC, Ludhiana) and Dr. Monika Sharma (Convener and In-charge ACME, Professor of Pediatrics, Vice Principal-Research, CMC, Ludhiana) in conducting this study. The authors also wish to thank the ACME faculty, CMC, Ludhiana, for guidance and support in conducting this study.
References
- 1.Joshi MK, Singh T, Badyal DK. Acceptibility and feasibility of mini clinical evaluation exercise as a formative assessment tool for workplace based assessment for surgical postgraduate students. J Postgrad Med. 2017;63:100–5. doi: 10.4103/0022-3859.201411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Tejinder S, Piyush G, Daljit S Assessment of Practical/Clinical Skills in Principles of Medical Education. Jaypee. (4th ed) 2013:123–35. [Google Scholar]
- 3.Tejinder S, Anshu The Mini-Clinical Evaluation Exercise (Mini-CEX) in Principles of Assessment in Medical Education. Jaypee. (1st ed) 2012:156–65. [Google Scholar]
- 4.Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: A method for assessing clinical skills. Ann Intern Med. 2003;138:476–81. doi: 10.7326/0003-4819-138-6-200303180-00012. [DOI] [PubMed] [Google Scholar]
- 5.Singh T, Sharma M. Mini-clinical examination (CEX) as a tool for formative assessment. Natl Med J India. 2010;23:100–3. [PubMed] [Google Scholar]
- 6.Behre R. Introduction of Mini-CEX in undergraduate dental education in India. Educ Health. 2014;27:262–8. doi: 10.4103/1357-6283.152187. [DOI] [PubMed] [Google Scholar]
- 7.Norcini JJ, Blank LL, Arnold GK, Kimball HR. The Mini-CEX (Clinical evaluation exercise): A preliminary investigation. Ann Intern Med. 1995;123:795–9. doi: 10.7326/0003-4819-123-10-199511150-00008. [DOI] [PubMed] [Google Scholar]
- 8.Goel A, Singh T. The usefulness of Mini clinical evaluation exercise as a learning tool in different pediatric clinical settings. Int J Appl Basic Med Res. 2015;5((Suppl 1)):S32–4. doi: 10.4103/2229-516X.162266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Liao KC, Pu SJ, Liu MS, Yang CW, Kuo HP. Development and implementation of a mini-Clinical evaluation exercise (mini-CEX) program to assess the clinical competencies of internal medicine residents: From faculty development to curriculum evaluation. BMC Med Educ. 2013;13:31. doi: 10.1186/1472-6920-13-31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Chang YC, Chen CK, Chen JC, Liao CH, Lee CH, Chen YC, et al. Implementation of the mini-clinical evaluation exercise in postgraduate year 1 residency training in emergency medicine: Clinical experience at Chang Gung memorial hospital. J Acute Med. 2013;3:110–5. doi: 10.1016/j.jacme.2013.06.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Ghadrdoost B, Sadeghipour P, Amin A, Bakhshandeh H, Noohi F, Maleki M, et al. Validity and reliability of a virtual education satisfaction questionnaire from the perspective of cardiology residents during the COVID-19 pandemic. J Educ Health Promot. 2021;10:291. doi: 10.4103/jehp.jehp_32_21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Holmboe ES, Yepes M, Williams F, Huot SJ. Feedback and mini clinical evaluation exercise. J Gen Intern Med. 2004;19:558–61. doi: 10.1111/j.1525-1497.2004.30134.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Burch VC, Seggie JL, Gary NE. Formative assessment promotes learning in undergraduate clinical clerkships. S Afr Med J. 2006;96:430–3. [PubMed] [Google Scholar]
- 14.Baraz S, Memarian R, Vanaki Z. Learning challenges of nursing students in clinical environments: A qualitative study in Iran. J Educ Health Promot. 2015;4:52. doi: 10.4103/2277-9531.162345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Malhotra S, Hatala R, Courneya CA. Internal medicine residents perceptions of Mini-Clinical Evaluation Exercise. Med Teach. 2008;30:414–9. doi: 10.1080/01421590801946962. [DOI] [PubMed] [Google Scholar]
- 16.Jafarpoor H, Hosseini M, Sohrabi M, Mehmannavazan M. The effect of direct observation of procedural skills/mini-clinical evaluation exercise on the satisfaction and clinical skills of nursing students in dialysis. J Educ Health Promot. 2021;10:74. doi: 10.4103/jehp.jehp_618_20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bhattacharyya H, Vagha J, Medhi GK, Pala S, Chutia H, Bora PJ, et al. Introduction of structured feedback for MBBS students: Perception of students and faculty. J Educ Health Promot. 2020;9:285. doi: 10.4103/jehp.jehp_406_20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Wellers JM, Jones A, Merry AF, Jolly B, Saunders D. Investigation of trainee and specialist reactions to the Mini Clinical Evaluation Exercise in anesthesia: Implications for implementation. Br J Anesth. 2009;103:524–30. doi: 10.1093/bja/aep211. [DOI] [PubMed] [Google Scholar]