An integrated mini-CEX and OSCE assessment model for optimizing standardized training in department of hepatobiliary surgery

Abstract

Background

Standardized residency training in China requires robust evaluation mechanisms, yet significant disparities exist due to lacking standardized criteria. Current tools like OSCE (simulation-based) and Mini-CEX (workplace-based) have limitations: Mini-CEX excludes surgical skills, while OSCE lacks authentic clinical context. Combining them offers holistic assessment potential but is understudied in hepatobiliary surgery.

Methods

A prospective cohort study randomized 36 hepatobiliary surgery residents into: Experimental group (n = 18): Monthly Mini-CEX assessments (evaluating medical interviewing, physical exam, clinical judgment, communication, treatment planning, patient-centered care, and overall competence via 9-point scale) followed by final OSCE. Control group (n = 18): Monthly traditional written/procedural assessments followed by final OSCE.

The OSCE (100-point max) comprised stations for history-taking, physical exam, hepatobiliary procedural skills, case analysis and doctor-patient communication. Intergroup comparisons used Student's t-tests and chi-square tests (SPSS 26.0, p < 0.05 significant).

Results

Baseline characteristics showed no significant differences between groups. The experimental group achieved significantly higher final OSCE total scores (p < 0.05), with notable improvements in physical examination and doctor-patient communication. No significant differences were found in history-taking, hepatobiliary procedural skills, or case analysis. Satisfaction was significantly higher in the experimental group (1/18 vs. 6/18 dissatisfied in controls). Trainees valued Mini-CEX's real-time feedback and competency mapping.

Conclusions

Integrating Mini-CEX and OSCE creates an effective dual-modality system for hepatobiliary surgery residents. It significantly enhances overall clinical competency (especially physical exam and communication) and trainee satisfaction compared to traditional methods. This combined approach provide both formative feedback (Mini-CEX) and summative validation (OSCE), establishing a promising specialty-specific framework. Future multi-center studies are recommended.

Graphical abstract

Highlights

• •Our integrated model merges Mini-CEX with OSCE, creating a comprehensive assessment that overcomes each tool's individual limitations for hepatobiliary surgery training.
• •Residents using this method achieved significantly higher OSCE scores, with marked improvement in physical examination and doctor-patient communication skills.
• •Trainee satisfaction was significantly higher, valuing the Mini-CEX for its real-time, competency-focused feedback within authentic clinical settings.

Introduction

Standardized residency training serves as an educational framework that integrates theoretical knowledge with clinical competencies. In 2014, the National Health Commission of the People's Republic of China formulated guidelines establishing a nationwide standardized training system for medical residents [1]. Recognized as the mandatory pathway for cultivating qualified post-graduate physicians, this system requires robust evaluation mechanisms to ensure training efficacy [2]. However, significant disparities persist in assessment protocols across different regions, primarily due to the absence of standardized criteria [3]. The development of a comprehensive, objective, and unified evaluation system therefore constitutes an imperative need in China's medical education reform.

Current clinical competency assessment tools in medical education include the Direct Observation of Procedural Skills (DOPS) [4], Mini-Clinical Evaluation Exercise (Mini-CEX) [4], and Objective Structured Clinical Examination (OSCE). Among these, the OSCE framework-first introduced by Harden et al. at the University of Dundee in 1975-utilizes simulated clinical scenarios to objectively evaluate medical trainees' knowledge and technical skills [5]. In contrast, the Mini-CEX, developed by the American Board of Internal Medicine, serves as a workplace-based assessment tool emphasizing real-patient encounters. While Mini-CEX offers practical and comprehensive evaluation for internal medicine residents, its exclusion of surgical skill assessment limits its applicability in surgical specialties [6], [7], [8], [9].

A critical gap persists in establishing a unified, specialty-specific evaluation system. Integrating Mini-CEX with OSCE demonstrates synergistic potential: OSCE's structured simulations address Mini-CEX's limitations in procedural skill assessment, while Mini-CEX's real-patient evaluations compensate for OSCE's lack of authentic clinical context. This dual approach enables holistic evaluation of residents' theoretical knowledge, practical skills, and clinical reasoning, effectively bridging the theory-practice divide to enhance comprehensive clinical competency.

Although combined Mini-CEX/OSCE models have shown promise in procedural specialties like obstetrics and dentistry [10], [11], their implementation remains understudied in general surgery, particularly in highly specialized subspecialties such as hepatobiliary surgery. This study proposes integrating Mini-CEX and OSCE to assess hepatobiliary surgery trainees, aiming to identify theoretical and technical deficiencies, to enhance operative skill development, and to establish a novel framework for cultivating surgical excellence. The findings may redefine competency-based training pathways for future surgical specialists.

Methods

Study subjects

This prospective cohort study enrolled 36 postgraduate trainees undergoing standardized residency training at the Department of Hepatobiliary Surgery, Nanjing Drum Tower Hospital, between January and December 2023. Participants were randomized into two equal cohorts: a control group (n = 18) and an experimental group (n = 18). Both groups attended the same departmental teaching conferences and grand rounds. They experienced identical clinical rotations with equivalent patient exposure and procedural opportunities within the hepatobiliary surgery department. The only deliberate difference was the structured, real-time feedback mechanism of the Mini-CEX for the experimental group versus the traditional written exam for the control group. The study protocol received approval from the Institutional Review Board (IRB) of Nanjing Drum Tower Hospital, and written informed consent was obtained from all participants prior to enrollment. Each participant completed a 3-month study cycle. During that cycle the experimental group received Mini-CEX assessments once per month (total of 3 Mini-CEX assessments per participant). The control group received one monthly traditional written and procedural skills assessment (total of 3 assessments). Both groups completed one 5-station OSCE at the end of the 3-month cycle (one OSCE per participant).

Assessing process

All participants were randomized into two groups (Fig. 1):

• •Experimental group (n = 18): Residents underwent monthly competency assessments using the Mini-CEX evaluation scale (Table S1). After 3 months, their clinical proficiency was re-evaluated using the OSCE examination system during graduation assessments.

  Experimental group (n = 18): Residents underwent monthly competency assessments using the Mini-CEX evaluation scale (Table S1). After 3 months, their clinical proficiency was re-evaluated using the OSCE examination system during graduation assessments.

• •Control group (n = 18): Residents received monthly evaluations via traditional written examinations and procedural skills assessments. Similarly, OSCE was administered after 3 months to assess graduation competency.

Fig. 1.

Satisfaction survey on training.

Mini-CEX scale

The Mini-CEX scale was developed to assess core clinical competencies of residents during real-patient encounters in hepatobiliary surgery training. This tool employs a 9-point scoring system categorized into three performance tiers:

• •Basic proficiency (needs improvement: 1–3)
• •Competent (meets expectations: 4–6)
• •Advanced (excellence demonstrated: 7–9)

Evaluation domains encompass seven key areas:

• 1.Medical interviewing
• 2.Physical examination
• 3.Clinical judgment
• 4.Patient communication
• 5.Treatment planning
• 6.Patient-centered care
• 7.Overall clinical competence

Supervisors provide real-time quantitative ratings and structured qualitative feedback during assessments. Immediately following the evaluation, formative feedback is delivered to trainees, emphasizing strengths and identifying areas for improvement. This process aims to refine clinical reasoning, enhance diagnostic decision-making, and optimize holistic patient management. All supervising assessors attended a standardized training session prior to the study. This session included: a) review of Mini-CEX and OSCE scoring rubrics with anchor examples, b) practice scoring using sample video scenarios, and c) training on delivering structured, constructive feedback.

OSCE scale

The OSCE assessment comprises five standardized stations tailored to hepatobiliary surgery:

• 1.Clinical history-taking
• 2.Physical examination
• 3.Hepatobiliary procedural skills
• 4.Case analysis
• 5.Doctor-patient Communication (Table S2)

  Doctor-patient Communication (Table S2)

Each station is scored on a 20-point rubric, yielding a maximum total score of 100. Performance is graded as:

• •Minimum competency: ≥60
• •Proficient: ≥75
• •Exemplary: ≥90

Evaluators score trainees based on predefined criteria at each station, ensuring objective measurement of technical skills, knowledge application, and clinical problem-solving abilities.

Satisfaction evaluation

All residents completed a six-item satisfaction survey (Table S3) assessing teaching methods and effectiveness. They rated each item as “Very satisfied,” “Satisfied,” or “Not satisfied” according to their perceived skill development and enjoyment of the training.

Statistical analysis

Data analysis was performed using SPSS 26.0 (IBM Corp., Armonk, NY, USA). Continuous variables are expressed as mean ± standard deviation (SD). Intergroup comparisons between the control and Mini-CEX groups were analyzed using Student's t-test. Categorical variables (e.g., satisfaction rates, presented as percentages) were evaluated with the chi-square (χ²) test. A two-tailed P-value ≤ 0.05 was defined as statistically significant.

Ethical consideration

The study was conducted only after approval from the ethical and scientific committee of our institution and after obtaining due consent from participating faculty and surgery residents.

Results

Characteristics of the residents

A total of 36 residents were included in this study, covering different grades and majors. Among them, 18 residents were in the experimental group and the remaining 18 residents were in the control group. The characteristics of all the residents are shown in Table 1. There were no significant differences between the general characteristics of the two groups (Table 1).

Table 1.

Comparison of experiment and control groups on baseline demographic variables.

	Experiment group	Control group	Sig.
Age (years)	26.83 ± 1.86	26.05 ± 1.83	0.214
Gender			0.711
Male	12	14
Female	6	4
Academic qualifications			0.551
BS	2	1
MM	11	14
MD	5	3
Specialty			0.850
General surgery	5	4
Urology	4	7
Neurosurgery	2	1
Cardiothoracicsurgery	3	2
Plasticsurgery	2	3
Orthopedics	2	1
Residency Training Grade			0.443
1	6	5
2	8	6
3	4	7

Note: BS: Bachelor of Science MM: Master of Science MD: Doctor of Science.

Comparison of experiment and control groups on OSCE scores

The experimental group demonstrated a statistically significant improvement in their final OSCE total score (84.78 ± 3.22) compared to the control group (81.11 ± 3.60, p < 0.01). Specifically, superior performance was noted in the “Physical examination” and “Doctor-patient Communication” stations. The differences were statistically significant. There was no significant difference between the two groups in history collection, hepatobiliary procedural skills and case analysis (P > 0.05). (Table 2).

Table 2.

Comparison of experiment and control groups on OSCE scores.

	Experiment group	Control group	p value
History taking	17.33 ± 1.37	16.61 ± 1.20	0.101
Physical examination	16.17 ± 0.79	15.11 ± 1.02	0.001
Hepatobiliary procedural skills	16.89 ± 0.90	16.39 ± 1.14	0.154
Case analysis	16.94 ± 1.00	16.72 ± 1.23	0.555
Doctor-patient Communication	17.44 ± 0.86	16.28 ± 0.89	<0.001
Total score	84.78 ± 3.22	81.11 ± 3.60	0.003

Resident satisfaction on training

Post-intervention satisfaction analysis revealed significant intergroup disparities in training receptivity. The experimental cohort demonstrated superior acceptance rates, with only 1/18 residents expressing reservations about training design, compared to 6/18 in controls who expressed dissatisfaction with the training plan (n = 2), whether the examiners interacted with the trainees (n = 2), whether the teaching content was coherent (n = 1), and whether their own practical abilities had improved (n = 1). Notably, 100% participants endorsed faculty pedagogical competence and training infrastructure adequacy across both groups (Fig. 2). The satisfaction survey particularly highlighted the Mini-CEX framework's dual strengths: a) real-time competency mapping through workplace-based assessment (WBA), and b) structured feedback loops enhancing clinical decision-making capabilities.

Fig. 2.

Flow chart of standardized residency training in department of hepatobiliary surgery.

Discussion

The fundamental goal of residency training evaluation is to guarantee that resident physicians achieve competency requirements in clinical skills, theoretical knowledge, and professional ethics through systematic, standardized assessments [12], [13]. This process ensures healthcare service standardization. Specifically, evaluation should encompass competency verification to confirm trainees' ability to independently diagnose and manage common diseases; feedback-driven improvement using formative assessments to identify knowledge gaps and enhance training programs; and quality assurance through standardized teaching management at training institutions to uphold educational quality. Ultimately, we aim to establish a competency-oriented evaluation system that drives pedagogical reform and cultivates clinical professionals who integrate clinical expertise with humanistic care.

Currently, most training bases face several significant challenges. Standards lack uniformity, with substantial disparities in assessment criteria across different bases. Daily assessments often become mere formalities in some locations, while end-of-rotation evaluations disproportionately emphasize theoretical knowledge at the expense of clinical reasoning. Furthermore, evaluation content remains incomplete by overlooking essential job competencies. Existing assessments primarily focus on theoretical knowledge and technical skills, failing to systematically evaluate core competencies such as communication skills, humanistic care, and research literacy. Consequently, implementing appropriate evaluation methods holds great significance for advancing medical education, improving training outcomes and enhancing medical ethics.

To address the aforementioned issues, we adopted the Mini-CEX scale as a formative assessment tool [14]. Compared with traditional evaluation methods, instructors provide real-time feedback and targeted guidance after each Mini-CEX assessment. In the final OSCE results, the experimental group outperformed the control group in all assessment categories, demonstrating the advantages of Mini-CEX. Notably, the Mini-CEX group showed significantly greater improvements in physical examination and doctor-patient communication skills, with statistically significant differences (P < 0.05). These two areas are precisely the components often neglected in traditional teaching methods, indicating that Mini-CEX better aligns with a clinical competency-oriented evaluation system. The reason could be that the Mini-CEX evaluation scale integrates concepts such as doctor-patient communication skills, humanistic care, and teamwork, thereby transforming the previous approach of merely assessing medical theoretical knowledge and practical skills.

Other literatures have also explored the effect of Mini-CEX combined with OSCE evaluation system. Martinsen SSS et al. demonstrated Mini-CEX's potential as a formative assessment tool for undergraduate medical students during clinical placements and routine practice [15]. Similarly, Luo et al. found that an OSCE and Mini-CEX integrated teaching model significantly enhanced surgical clerks' self-confidence and clinical competence [16]. However, these studies primarily involved medical trainees such as interns and students. Research specifically examining the impact of combining Mini-CEX with OSCE on standardized surgical residents remains limited. Our study further confirms the effectiveness of the combined Mini-CEX and OSCE approach within the standardized training program for hepatobiliary surgery.

Overall, residents responded positively to the novel assessment model. As research indicates that feedback quality significantly enhances an assessment tool's utility [17], this model facilitated both constructive and critical input. Only one participant in the experimental cohort (1/18) raised concerns about the training design, citing the relatively long duration of each Mini-CEX assessment. Consequently, we are considering reducing the time allocated for history-taking in subsequent implementations.

While this study demonstrates promising educational outcomes, several methodological constraints warrant careful interpretation of the findings. The modest cohort size (n = 18) may have reduced statistical power to identify clinically meaningful effect sizes. In the future, we are considering including students from multiple centers. A multi-center study would significantly enhance the generalizability of our findings. Implementing external evaluators would be a crucial methodological improvement. This would effectively eliminate potential assessment bias, thereby strengthening the validity of the outcome measures. Furthermore, the single-institutional setting inherently incorporates site-specific biases in patient demographics, surgical case complexity, and institutional culture that may limit extrapolation to diverse healthcare ecosystems. Future multi-center studies with stratified sampling are recommended to validate these preliminary observations.

Notwithstanding these limitations, our integrative assessment model synergizing Mini-CEX and OSCE establishes a bi-functional paradigm in surgical education-serving simultaneously as a developmental scaffolding mechanism and summative competency validator. Specifically tailored to hepatobiliary surgical training exigencies, this dual-modality approach demonstrated discriminant validity in evaluating the operation ability and actual diagnosis and treatment ability of graduate students respectively. Hence, it is expected to prepare the residents to deal with patients in a better way in the future.

CRediT authorship contribution statement

Hucheng Ma: Writing – original draft, Validation, Methodology, Investigation, Conceptualization. Haozhen Ren: Writing – original draft, Methodology. Xingyu Wu: Writing – review & editing, Validation, Methodology. Decai Yu: Writing – review & editing, Validation, Supervision, Conceptualization.

Ethics approval

Authors followed ethics guidelines provided by Elsevier.

Funding

Project of China Hospital Reform and Development Research Institute, Nanjing University, NDYGN2023083.

Declaration of competing interest

The authors report no proprietary or commercial interest in any product mentioned or concept discussed in this manuscript.

Appendix A. Supplementary data

Table S1

Mini-CEX assessment form for department of hepatobiliary surgery.

Table S2

OSCE scale for evaluating specific skills in residents of hepatobiliary department.

Table S3

Satisfaction survey on the assessment of resident standard training in department of hepatobiliary surgery.