Evaluation of mini-clinical evaluation exercise (mini-CEX) as a method for assessing clinical skills in anaesthesia postgraduate education

INTRODUCTION

Clinical skills are most essential for patient care; however, the assessment of clinical competencies can be challenging and complex. Currently, most traditional assessment methods for postgraduate students (PGS) used in most medical colleges in India (yearly written essay type questions and summative ratings including oral case-based discussions and table viva voce) are summative and try to find out how much the PGS gained knowledge rather than their clinical ability.[1]

Nowadays, there is a trend toward competency-based training with emphasis on formative rather than summative assessment. In India, the new undergraduate (UG) curriculum is competency- based. Workplace-based assessment (WPBA) is integral to programmatic assessment in a competency-based curriculum.[2] WPBA involves two things: (1) Direct observation of clinical and non-clinical skills in real situations especially the workplace.(2) Proper feedback by evaluator to trainee. Timely feedback contributes to student learning and improvement and is the heart of medical education. It is said that 80% PGS in India do not receive any feedback. There are several methods of WPBA. Mini-clinical evaluation exercise (mini-CEX) is one method of formative assessment and WPBA. In mini-CEX, an expert, usually a faculty member observes the actual encounter of the trainees with the patient at the workplace, rates their history taking, physical examination and counselling skills and provides feedback to them.[3] There have been some studies in western countries and a few from India on the use of mini-CEX in PG medical education. Few non-anaesthesia related studies on this topic have been published from India.[4,5,6] The mini-CEX has been adopted into anaesthesia training programmes in some countries.[2,7,8] We conducted a study on PGS in anaesthesia with an aim to evaluate mini-CEX as a method for assessing clinical skills in anaesthesia PG education. Our primary objectives were to assess the perceptions of PGS and faculty members regarding mini-CEX as an assessment/learning tool in Anaesthesiology and Critical Care. The secondary objectives were to introduce, sensitise, and train our faculty and PGS in the use of mini-CEX.

METHODOLOGY

Institutional ethics committee approval was obtained. The cross-sectional study took place from November 2019 to March 2020 in the Department of Anaesthesiology at a government medical college in our country. The study population included PGS and faculty members. Written informed consent was taken from the participants. Initially, the students and faculty were sensitised by us regarding mini-CEX with the help of a short talk and a video session on how the exercises would be conducted and what was expected out of them. Trainee-patient encounters with different topics and scenarios were planned in anaesthesiologist-specific workplaces : pre-anaesthesia examination (PAE) clinic, the pain and palliative care clinic, postoperative ward, and the surgical intensive care unit (ICU) [Table 1]. The lists of faculty and student names were displayed on the notice-board and in faculty and PGS Whatsapp groups. In an encounter, the PGS was given an exercise on history taking, physical examination, and counselling. The student was instructed beforehand that he or she was being observed by the faculty member, that his performance was being scored and the time allotted to him. Observation by the faculty was done for 15 min and data was entered in the mini-CEX form [Annexure A]. This was followed by an oral feedback for 5 min by the observer faculty to the student. The assessment was thus done in routine working conditions which is not usually done at assessment during standard viva examinations. At the end of the encounter, a prevalidated questionnaire was given to the student regarding the conduct of mini-CEX [Annexure B]. At the end of the study, the participating faculty and PGS were given questionnaires dealing with their perceptions on the value and acceptance of mini-CEX as an assessment tool in anaesthesiology. The questionnaires were based on existing studies investigating faculty and student views on mini-CEX.[5,9,10] The PGS maintained anonymity while filling the responses. The questionnaires were prevalidated by subject experts as well as experts in medical education from the authors’ institutes.

Table 1.

Examples of clinical encounters that were used for mini-clinical evaluation exercises

Level of PG student	Example of the types of encounters for mini-CEX
PG (1st year)	Pre-operative assessment of adult patient for elective surgery
	Patient consent for spinal/epidural anaesthesia at pre-operative assessment
	Patient consent for central vascular access without GA
PG (2nd year)	Assessment of cancer patient with pain
	Pre-operative assessment of adult/paediatric patient for elective surgery
	Assessment of case for emergency caesarean section
	Assessment of patient for elective caesarean section
	Assessment of head injured and trauma patient on mechanical ventilation in ICU
	Postoperative pain assessment
PG (3rd year)	Assessment of a moribund patient
	Pre-anaesthetic assessment of obstetric patient for laparoscopic procedure
	Assessment of postoperative care in ICU (patient not extubated)
	Pre-operative assessment of neonate for elective anaesthesia
	Pre-operative assessment of adult for emergency laparotomy
	Assessment and counselling of brain-dead patients in ICU
	Assessment of cancer patient with pain
	Family counselling in ICU
	Pre-operative assessment of neonate for emergency surgery
	Nutritional counselling in the ICU

ICU – Intensive Care Unit; GA – General anaesthesia; PG – Postgraduate

Quantitative data from the responses to the questionnaires were analysed by using the Statistical Package for the Social Sciences (SPSS) version 20 (International Business Machine, United States).

RESULTS

Twenty-nine faculty-PGS encounters including 20 faculty members and 27 PGS (11: 1^st year, 14:2^nd year, 2: 3^rd year) took place.

The total number of PGS exhibiting satisfactory, superior, and unsatisfactory scores for different clinical skills was calculated.

As regards the conduct of mini-CEX, 100% PGS felt that the topics selected were useful, pre-encounter instructions given were clear and adequate, useful feedback was given and the exam was well- coordinated and organised. 52% PGS felt that the examiner was intimidating, 15% felt that personality and gender did affect the conduct of mini-CEX, 2% encountered multiple disturbances during the exercise, 2% felt that the time allocated for the exercise was not sufficient, and 8% felt that the questions asked were not relevant.

A majority of the students had positive perceptions regarding various aspects of mini-CEX as an assessment tool in anaesthesia [Figure 1a]. The questionnaire on faculty perceptions regarding mini-CEX as an assessment tool revealed positive responses for a majority of the questions except for the question asking whether mini-CEX eliminates examiner bias [ [Figure 1b]. The mean satisfaction score with mini-CEX was 7.28 (range 5-9) for faculty and 8.08 (range 6-10 ) for PGS. Scores were superior in 60% PGS for clinical judgement, 62% PGS for counselling skills, 44% PGs for organisation and efficiency, 62% PGS for professionalism and 60% PGS for overall clinical competence.

Figure 1.

Perceptions regarding mini-CEX as an assessment tool in anaesthesiology (a) Student perceptions, (b) Faculty perceptions

DISCUSSION

The responses and scores show that a majority of the PGS and faculty in our study were satisfied with the use and conduct of mini-CEX as an assessment tool in anaesthesia. This shows that they perceived mini-CEX as an effective and efficient learning tool in anaesthesiology. A meta-analysis has demonstrated the positive effects of mini-CEX on trainee performance.[11] The one-to-one interaction with faculty in authentic clinical settings, case diversity, and multiple encounters with different assessors in mini-CEX has already been welcomed by residents and faculty in specialities other than anaesthesia.[4,5,6] Mini-CEX has been perceived in another study to have very positive educational impacts and relative feasibility among anaesthesia trainees.[12]

The Mini-CEX assessment tool has several weaknesses too including creation of teacher-student friction, demoralisation of students, variable assessor stringency, assessor subjectivity, and case-specificity.[12] The students in our study were a bit nervous, anxious, and hesitant when they came to know that their chance had come for the encounters; however, this was only for the first few encounters. Joshi et al. have also reported nervousness while examining cases in some of their study population.[10] Castanelli et al. found some difficulties in finding time to schedule assessments and deliver timely feedback in busy clinical workplaces.[2] We had planned and organised the encounters in such a way that no busy work schedule was hampered. Our study has some limitations. The number of encounters is less and it is a single-centre study; nevertheless, based on this pilot study, we propose to conduct a larger study in the future.

We conclude that mini-CEX is an effective formative assessment tool of potential value to assess clinical skills in anaesthesiology. It can be incorporated into anaesthesia PG education curriculums in India and elsewhere.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Annexure A.

The mini-Clinical evaluation exercise (mini-CEX) form

Evaluator (faculty):_____________________________										Date: ____________________
Postgraduate student (PG)_________________________								oR-1	oR-2	oR-3
Patient Problem/Dx
Setting:		PAE room:		O.T	SICU			Post-op ward	Pain and palliative care clinic
Patient:		Age:__________		Sex:__________				o New		o Foldlow-up
Focus: oData gathering oDiagnosis				oTherapy	oCounselling-Patient___Relatives____					o Giving preop instruction
1. Medical interviewing skills (o Not observed)
1	2	3	4	5	6			7	8	9
Unsatisfactory				Satisfactory					Superior
2. Physical examination skills (o Not observed)
1	2	3	4	5	6			7	8	9
Unsatisfactory					Satisfactory					Superior
3. Humanistic qualities/Professionalism (o Not observed)
1	2	3	4	5	6			7	8	9
Unsatisfactory				Satisfactory					Superior
4. Clinical judgement (o Not observed)
1	2	3	4	5	6			7	8	9
Unsatisfactory				Satisfactory					Superior
5. Counselling skills (o Not observed)
1	2	3	4	5	6			7	8	9
Unsatisfactory				Satisfactory					Superior
6. Organisation/efficiency (o Not observed)
1	2	3	4	5	6			7	8	9
Unsatisfactory				Satisfactory					Superior
Overall clinical competence (o Not observed)
1	2	3	4	5	6			7	8	9
Unsatisfactory				Satisfactory					Superior
Mini-CEX time:		Observing: _____min			Providing feedback: _____min					Total score:_____
Evaluator (Faculty) satisfaction with mini-CEX
Low	1	2	3	4	5	6	7	8	9	High
PG student satisfaction with mini-CEX
Low	1	2	3	4	5	6	7	8	9	High
Comments:
________________________					_________________________
PG signature					Evaluator signature

Dx: Diagnosis; PAE: Pre anaesthetic evaluation; R-1=first year PG; R-2=second year PG; R-3=third year PG; OT: Operation theatre; SICU: surgical intensive care unit; post-op: postoperative; preop: preoperative

Annexure B.

Questionnaire for student's perceptions regarding conduct of mini-CEX in anaesthesiology

Item No	Items	Yes	No
1	The questions asked were relevant to the situation
2	The topics and activities selected for the mini-CEX encounters were relevant
3	Instructions given before encounter were clear and adequate
4	Time allocated was sufficient at each encounter
5	I was troubled multiple times the assessment was being conducted
6	Personality, gender and other attributes did not affect the mini-CEX scores
7	Examiners (observer faculty) were intimidating
8	Useful and motivating feedback was given
9	The entire exam was well co-ordinated and organised