Introduction of Mini Clinical Evaluation Exercise as an Assessment Tool for M.B.B.S. Interns in the Department of Orthopaedics

Abstract

Introduction

Mini-CEX helps in judicious use of competencies in authentic settings by simultaneously assessing clinical skills of trainees and providing feedback on their performance. As assessment of M.B.B.S. Interns for their competency in clinical examination skills in Department of Orthopaedics is lacking, this study is taken up to introduce Mini-CEX for M.B.B.S. Interns, by sensitising faculty and interns.

Materials and Methods

A Quasi-experimental study was conducted during June to December 2020, among 60 interns posted in the Department of Orthopaedics. After obtaining IEC Clearance and written informed consent from the study participants, they were sensitised and exposed to five Mini-CEX clinical encounters involving examination of a patient with knee/other joint disorder in the Outpatient/Inpatient clinical setting, with eight faculty. The study tool used was Mini-CEX questionnaire developed by American Board of Internal Medicine (ABIM). Case specific feedback was provided to interns using sandwich technique. The reflections and perceptions of interns and faculty were obtained after completion of all Mini-CEX encounters.

Results

96.7% encounters were conducted in OPD during first encounter. On an average, one Mini-CEX encounter lasted for 17 min. Interns had an overall score for the various domains ranging from 5.38 to 5.58. Comparison of mean scores showed a statistically significant improvement (p value < 0.0001). All the assessors were satisfied with Mini-CEX as an assessment tool.

Conclusion

Interns and faculty opined that Mini-CEX improves clinical examination skills and professional development as focus is on the outcome and learning process, with multiple sampling in a longitudinal manner.

Introduction

Mini-CEX is a workplace-based assessment that conforms to the highest level of Miller’s Pyramid. It is a method for simultaneously assessing the clinical skills of trainees and offering them feedback on their performance [1]. The traditional system of assessment measures the outcome rather than the learning process, but Mini-CEX focuses on the outcome and learning process, with multiple sampling in a longitudinal manner [2]. It helps to provide learner feedback which drives learning by high-quality constructive feedback to improve clinical competency [3]. Mini-CEX helps to assess the history, physical examination, professionalism, clinical judgement, counselling skills and organisational efficiency in an intern during each clinical encounter and in this way it is different from the other types of workplace-based assessments. It provides for assessment of a trainee several times by a faculty on these domains during clinical posting. Use of multiple examiners for multiple cases helps to overcome interrater bias.

It is reported that there is no direct observation by faculty during patient encounters due to lack of faculty time [4]. Direct observation is important for the assessment of clinical skills including interviewing and counselling skills [5]. Assessment of competency in clinical examination skills among M.B.B.S. Interns in the Department of Orthopaedics is lacking in the present system of the internship. Hence, an attempt has been made in this study to introduce Mini Clinical Evaluation Exercise (Mini-CEX) as an assessment tool for MBBS interns, by sensitising the faculty and interns and to analyse their perception and feedback on implementation of the same in the Department of Orthopaedics.

Materials and Methods

This study was a Quasi-experimental study conducted during June to December 2020, among interns posted on rotation in the Department of Orthopaedics of our institute. Scientific Research Committee approval (vide letter no. SRC NO:09/April/2020 dated 30/04/2020), Institutional Ethical Clearance Certificate Clearance (vide letter no—IEC/2020/045) and written informed consent from the study participants were obtained after briefing them about the purpose and nature of this research study. Core Project Committee was constituted. Sixty MBBS interns selected by convenient sampling technique were included in the study.

Inclusion Criteria: the interns, Department of Orthopaedics, who gave consent to the study.

Exclusion Criteria: the interns posted in other departments.

Mini-CEX was introduced as a formative assessment tool after sensitisation and training of the faculty. Each intern was exposed to five encounters in the clinical examination of the knee/other joint. The Mini-CEX tool developed by American Board of Internal Medicine (ABIM, 2007) was used for direct observation to rate a total of 300 Mini-CEX encounters assessed by 8 faculty assessors. It is a reliable, structured and validated observation tool consisting of history taking, physical examination, professionalism, clinical judgement, counselling skills and organisational efficiency as domains to be assessed [6]. The assessment is discussed with the trainee, who signs the form at the end of this discussion. During every encounter, the teacher notes the date, complexity of the case (low, average, high), the type of visit (new or follow-up), the setting (ambulatory, inpatient, outpatient), the number of minutes spent observing the encounter and the number of minutes spent giving feedback. The validated faculty feedback and the intern feedback questionnaires were used to obtain feedback from faculty and interns. Data collected was subjected to both quantitative and qualitative analysis. Descriptive statistical analysis was done using the MS Excel 2007 and in IBM SPSS version 21, and mean (standard deviation) and median (interquartile range) were calculated. Comparison of mean scores obtained in the first and fifth encounter was done to see if there is any improvement. Difference in scores of various domains in Mini-CEX among study subjects across multiple encounters was tested for statistical significance, using Friedman’s ANOVA test for repeated measures, p value < 0.05 was considered statistically significant. Comparison of the mean time taken for observation and feedback was calculated. Mean satisfaction scores of the interns with the assessment using Mini-CEX was calculated. Qualitative analysis was done to identify themes for the responses to open-ended questions and tabulated for the open feedback. Feedback, both quantitative and qualitative, was collected from the faculty and interns by the feedback questionnaire. Mean satisfaction scores in each of the Mini-CEX encounters were also calculated.

Results

A total of 60 interns posted in the Department of Orthopaedics, on rotation, consented to participate in the present study. Out of 60, 20 (33.33%) were male interns while the remaining 40 (66.66%) were female. The clinical setting where the encounters were conducted was as follows. 96.7% were in OPD during the first encounter while in the fifth encounter, there were 78.3% in OPD. 100% were knee joint cases in first encounter whereas 81.7% were knee joint cases in fifth encounter. 91.7% were new cases during the first encounter while it was 50% in fifth encounter. In the first encounter, 81.7% cases were of average complexity while it was 36.7% in fifth encounter. In first clinical encounter, focus of domain was history taking in 55 (91.7%), in second clinical encounter, it was communication skills in 37 (61.7%), in third clinical encounter, focus of domain was clinical judgement in 40 (66.7%), in fourth one, it was professionalism in 35 (58.3%) and in the fifth one, it was organisational efficiency in 27 (45%) and overall clinical care in 60%.

As seen in Fig. 1, the comparison of mean scores obtained in first Mini-CEX clinical encounter and fifth encounter shows improvement. Difference in scores of various domains in Mini-CEX among study subjects as depicted in Table 1, shows that there is statistically significant improvement across multiple encounters, using Friedman’s ANOVA test for repeated measures (p value-0.0001).

Fig. 1.

Comparison of mean scores between the first and fifth encounters. As seen in Fig. 1, the comparison of mean scores obtained in first Mini-CEX clinical encounter and fifth encounter shows improvement.

Table 1.

Scores obtained for various domains in Mini-CEX among study subjects

Domain	1		2		3		4		5		p value
	Mean (SD)	Media n (IQR)	Mean (SD)	Me dia n (IQR)	Mean (SD)	Medi an (IQR)	Mea n (SD)	Medi an (IQR)	Mea n (SD)	Medi an (IQR)
History taking skills	3.9 (1.05)	4 (4–4)	4.25 (0.94)	4 (4–5)	4.73 (0.72)	5 (4–5)	5.13 (0.61)	5 (5–6)	5.58 (0.67)	6 (5–6)	0.0001
Physical examina tion skills	3.91 (1.03)	4 (4–4)	4.11 (0.93)	4 (4–5)	4.63 (0.85)	5 (4–5)	5.1 (0.74)	5 (5–6)	5.46 (0.71)	6 (5–6)	0.0001
Commun ication skills	3.93 (1.13)	4 (4–4)	4.31 (0.93)	4 (4–5)	4.68 (0.88)	5 (4–5)	5.05 (0.78)	5 (4.25–6)	5.51 (0.71)	6 (5–6)	0.0001
Clinical judgeme nt	4.00 (1.23)	4 (4–4)	4.16 (0.96)	4 (4–5)	4.66 (0.69)	5 (4–5)	5.01 (0.67)	5 (5–5)	5.50 (0.71)	6 (5–6)	0.0001
Professi onalism	4.05 (1.13)	4 (4–5)	4.15 (1.02)	4 (4–5)	4.66 (0.81)	5 (4–5)	5.06 (0.74)	5 (4.25–6)	5.38 (0.71)	6 (5–6)	0.0001
Organisa tional efficienc y	3.95 (1.10)	4 (4–4)	4.26 (0.94)	4 (4–5)	4.56 (0.80)	5 (4–5)	5.08 (0.69)	5 (5–6)	5.41 (0.64)	5.5 (5-6)	0.0001
Overall clinical care	4.96 (1.01)	4 (4–5)	4.23 (1.00)	4 (4–5)	4.68 (0.69)	5 (4–5)	5.08 (0.61)	5 (5–5)	5.46 (0.74)	6 (5–6)	0.0001

Difference in scores of various domains in Mini-CEX among study subjects as depicted in Table 1, shows that there is statistically significant improvement across multiple encounters, using Friedman’s ANOVA test for repeated measures (p value-0.0001)

The average time for first encounter was 15 min while for the fifth one, it was 20 min. Mean satisfaction scores of assessors were found to be 6.56 in the first encounter and 8.28 in the fifth one. Mean satisfaction scores of interns were found to be 5.9 in first encounter and 7 in the fifth one. 70 to 75% of the interns strongly agreed that it was better than conventional case taking. 23.3% opined that they were anxious during the clinical encounter. All of the faculty (100%) felt that there was a definite improvement in the clinical examination skills in all the domains during the Mini-CEX encounters, which was a feasible exercise in the department and an effective method to reduce the stress of exams.

The main themes that emerged during qualitative analysis of faculty feedback were enthusiastic participation of interns and patients as facilitating factors of this study, busy schedules due to covid-19 as barriers and for better implementation of Mini-CEX in the future, it was opined that it has to be incorporated into Final MBBS Part-II curriculum. Interns opined that they were motivated to perform better during each of the subsequent encounters because of the timely faculty feedback.

Discussion

The results of this study suggest that it was helpful to bring out the objectives stated, hence strengthen the need to include Mini-CEX as an assessment tool for interns to develop their clinical skills. In study by Balakrishnan R Nair et al., average complexity of encounters was found in 150 encounters and the average Mini-CEX encounter lasted for 16 min while, in the present study, it was of average complexity in 192 encounters and it lasted for 17 min. Almost all the assessors were satisfied with the Mini-CEX as an assessment tool which was consistent with the present study [7]. According to Atul Goel et al., the overall average score for all students was 5.65 while, in the present study, it was 5.38–5.58 for various domains [8].

Qualitative analysis showed that most of the interns in the present study, opined saying “It was an interesting experience because of the immediate feedback”. Positive effects of narrative feedback have been reported by various authors like Overeem [9] who found higher satisfaction with such feedback, and Govaerts [10] who suggested that narrative feedback can improve in training evaluation.

Conclusion

The interns and the faculty considered Mini-CEX as a feasible exercise and were highly satisfied with Mini-CEX tool. It was perceived that Mini-CEX is a feasible instrument where patient interviewing skills can be taught as part of actual patient care. Interns opined that immediate feedback after each encounter motivated them to perform better in the subsequent ones.

Limitations

Complexity of patient problems, focus of each encounter and relative amount of time spent per encounter and for providing feedback are some issues to be addressed for routine use of Mini-CEX.

Funding

Nil.

Data availability

The data will be made available on request.

Declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

Scientific Research Committee approval (vide letter no. SRC NO:09/April/2020 dated 30/04/2020), Institutional Ethical Clearance Certificate Clearance (vide letter no—IEC/2020/045) and written informed consent from the study participants was obtained after briefing them about the purpose and nature of this research study.

Informed Consent

For this type of study, informed consent is not required.

Participants' consent

The author certifies that participant consent forms were obtained. In the form, the participants have given their consent to take part in the study for the stated study purpose, without expecting any incentive in return. The participants understand that their participation is voluntary and that they can withdraw at any time.