Developing a Framework of Entrustable Professional Activities for Residency Training in Hematology

Abstract

Purpose

Entrustable professional activities (EPAs), essential for competent and safe clinical practice, serve as milestones to assess trainees’ readiness for independent patient care. This study aimed to develop a context-specific EPA framework for hematology residency training in China.

Methods

We employed a modified Delphi method involving 35 experts from 8 academic hospitals to establish China’s inaugural hematology-specific EPA framework. The development process comprised three key components: (1) a systematic literature review integrating international EPA models with China’s the Standardized Training System for Residents (STSR) requirements; (2) structural validation through the EQual rubric (cut-off: 4.07); and (3) a two-round Delphi consultations evaluating importance, observability, evaluability, repeatability, feasibility and entrustment-supervision levels. Expert engagement metrics and inter-rater agreement evaluated consensus quality.

Results

The preliminary 14-item framework was ultimately refined to 12 EPAs following the exclusion of “Public Health Emergency Response” (EPA 14; mean score=3.23) and “Imparting Bad News” (EPA 13; 57.1% experts advocated removing). Key revisions involved expanding the hematopoietic stem cell transplantation (HSCT) competency domain to include donor selection (EPA 7) and the provisional addition then subsequent removal of “Cellular Immunotherapy Complications Management” due to insufficient standardization of assessment criteria. The final EPAs achieved unanimous EQual compliance (≥4.07) and strong Delphi consensus (Kendall’s W: Round 1 = 0.271, Round 2 = 0.529, p<0.001). Clinical entrustment-supervision levels were systematically stratified by training phase and expected competency at graduation.

Conclusion

This study has established China’s first hematology-specific EPA framework by addressing gaps in global EPA models through integration of integrating local clinical demands (high patient volume, resource constraints) and multidisciplinary coordination. The framework provides a validated tool for competency-based assessment, with future steps focusing on digital implementation and nationwide validation.

Introduction

The global paradigm of medical education has evolved from the Flexner model to a competency-based approach, driven by the need to align training with societal healthcare demands.¹ While competency-based medical education (CBME) frameworks emphasizing predefined competencies have been widely adopted globally,² significant challenges persist in translating abstract competencies into measurable clinical assessments.³ This critical gap led to the development of entrustable professional activities (EPAs) by Olle ten Cate in 2005,⁴ which bridge theory and practice by defining discrete, observable clinical tasks residents must perform autonomously.⁵

Since their introduction, EPAs have been successfully integrated across undergraduate and postgraduate medical education,⁶ with validated frameworks existing for multiple specialties including pediatrics,⁷ anesthesiology,⁸ psychiatry,⁹ obstetrics/gynecology¹⁰ and nursing.¹¹ However, hematology - a discipline requiring mastery of both complex procedures (eg, bone marrow interpretation) and urgent clinical decision-making (eg, managing coagulopathies) – notably lacks a standardized EPA framework for residency training, despite its critical need for competency-based assessment tools.

China’s Standardized Training System for Residents (STSR), implemented since 2010, represents a significant shift toward competency-based medical education in China.¹² This mandatory three-year program requires residents to rotate through specialties such as hematology, aiming to standardize clinical proficiency. However, critical gaps persist in core competency assessment, particularly in: (1) analytical integration (eg, correlating bone marrow findings with clinical decision-making); (2) time-sensitive management of hematologic emergencies (eg, acute transfusion reactions); and (3) multidisciplinary care coordination. These assessment deficiencies are exacerbated by structural constraints including excessive clinical workloads and compressed rotation schedules in Chinese teaching hospitals.¹³

The current STSR assessment tools demonstrate limited reliability in evaluating these competencies,^14,15 especially for high-stakes scenarios such as diagnosis of hematologic malignancies, management of chemotherapy complications, and development of hematopoietic stem cell transplantation protocols. This underscores the urgent need for developing a hematology-specific EPA framework tailored to China’s unique clinical training environment.

This study bridges this gap through the development of China’s first hematology-specific EPA framework for internal medicine residency training in China. Through Delphi consultations and validation via the EQual rubric, we identified core EPAs essential for hematology residency training. Our framework aims to standardize competency assessments and enhance the reliability of entrustment decisions in high-acuity clinical settings.

Materials and Methods

Study Design

This mixed-methods study employed a modified Delphi method to develop and validate a hematology-specific EPA framework for residency training in China. The research process comprised three phases (Figure 1): (1) systematic literature review and preliminary EPA list development, (2) structural validation employing the EQual rubric, and (3) a two-round Delphi consultations.

Figure 1.

Represent the design and flow of the current study.

Composition of the Research Team

The research team comprised seven hematologists, including an education department director, a ward director and five senior clinical front-line teachers, along with three medical education administrators specializing in competency-based curriculum design. All members held senior professional titles (professor/associate professor) and advanced degrees (8 PhDs and 2 master’s degrees), with demonstrated publication records in either medical education or hematology research. The team supervised all aspects of expert recruitment, questionnaire design, data collection, and qualitative analysis.

Preliminary EPA Development

We conducted a literature review to identify relevant essential clinical activities suitable for inclusion as EPAs in China’s hematology residency training. We searched four electronic databases (Pubmed, Medline, Embase and Web of Science) in July 2024 for publications from 2012 to 2024 using the following keywords: “entrustability and hematology” and “entrustable professional activities and hematology”. After screening 132 records, we selected 14 studies to inform the draft EPA list. Building upon international EPA frameworks, including the AAMC Core EPAs, the Royal College of Physicians and Surgeons of Canada’s 20 EPA model, and the ASH Hematology-Oncology Curricular Milestones, we integrated China’s 2020/2022 residency training standards to develop a preliminary competency-based EPA list (Table 1). This EPA framework was designed to align with the three-year residency rotation structure, facilitating ensuring progressive entrustment from foundational skills (Year 1) to complex clinical decision-making (Year 3).

Table 1.

Validation of Initial EPAs Using EQual Score

EPA	Titles	Domain-Specific Average Scores			Overall Score Average	Overall Score Range	No. of Experts Recommending Revision (%)
		Discrete Units of Work	Entrustable	Curricular Role
EPA 1	Diagnosing and managing common myeloid hematological disorders	5	5	5	5	-	0
EPA 2	Diagnosing and managing common erythroid hematological disorders	5	5	5	5	-	0
EPA 3	Diagnosing and managing common lymphocyte and plasma cell disorders	5	5	5	5	-	0
EPA 4	Diagnosing and managing common hemorrhagic and thrombotic disorders	5	5	5	5	-	0
EPA 5	Diagnosing and managing common myeloproliferative neoplasms	5	5	5	5	-	0
EPA 6	Making differential diagnosis of cytopenia resulting from non-hematological diseases	4.73	4.80	4.80	4.77	4.20–5.00	1 (20)
EPA 7	Assessing the indications and contraindications for Hematopoietic Stem Cell Transplantation (HSCT)	4.67	4.75	4.65	4.69	4.00–5.00	2 (40)
EPA 8	Recognizing and managing chemotherapy-related toxicities	4.80	4.90	4.95	4.87	4.40–5.00	1 (20)
EPA 9	Recognizing and managing febrile neutropenia	4.77	4.75	4.80	4.77	4.20–5.00	0 (0)
EPA 10	Recognizing and managing blood transfusion reactions	4.80	4.85	4.85	4.83	4.60–5.00	0 (0)
EPA 11	Recognizing and managing bleeding	4.77	4.85	4.85	4.81	4.60–5.00	0 (0)
EPA 12	Performing procedures for the evaluation of hematological disorders	5	5	5	5	-	0
EPA 13	Imparting bad news	3.77	3.95	3.90	3.86	3.40–4.40	4 (80)
EPA 14	Public health incident response	3.13	3.30	3.30	3.23	2.60–3.80	5 (100)

EQual Rubric Validation

To ensure methodological rigor, the preliminary EPA framework underwent quality validation using the EQual rubric¹⁶ —a structured tool designed to identify content gaps and structural inconsistencies. Five external medical education experts (median experience: 12 years; IQR: 10–15) independently evaluated the draft EPAs using the 14-item EQual rubric. Each item was scored on a 5-point Likert scale (1=strongly disagree; 5=strongly agree). An Angoff-modified cutoff score of 4.07 was applied, with EPAs scoring below this threshold undergoing revision. Open-ended feedback addressed content gaps and alignment with clinical practice. The number of experts recommending revision for each EPA was documented and compared with its performance on the EQual rubric relative to the overall cutoff.^16,17

Expert Recruitment

The experts were selected according to the following selection criteria: 1) active involvement in standardized residency training programs with demonstrated capabilities in clinical teaching, educational assessment, process management and medical education research; 2) over ten years of clinical work and teaching experience; 3) physical attendance at EPA training sessions conducted by the research team. Exclusion criteria consisted of: 1) inability to comprehend English; 2) unavailability to complete surveys or attend in-person EPA training sessions. Participants were invited by email, and informed consent was obtained from each volunteer. The research team then conducted an orientation session to ensure conceptual alignment, which included presentations on EPA fundamentals, study objectives, background information, and a preliminary evaluation index system. Participants were asked to rate each indicator’s the importance and their personal familiarity level.

Implementing Consensus Method to Establish an EPA-Framework

We employed a modified Delphi method to evaluate EPAs in hematology for residents.¹⁸ The first-round Delphi questionnaire was administered to the expert panel, focusing on five key dimensions for each indicator: importance, observability, evaluability, repeatability and feasibility. Fourteen EPA items were scored using a 5-point scale (1 = lowest, 5 = highest). Experts graded the entrustable level that should be achieved, referring to the fivelevel entrustment-supervision scale proposed by ten Cate, which were enriched in additional levels by Chen.¹⁹ The scale defines entrustment-supervision scale: (1) No permission to practice EPA (1a: insufficient knowledge/skills without observation rights; 1b: permitted observation with basic knowledge); (2) Full supervision requiring direct trainer involvement (2a: co-activity performance; 2b: supervisor-ready backup); (3) Conditional supervision with tiered oversight (3a: immediate availability with all findings double checked; 3b: key findings double checked; 3c: distant availability with retrospective review); (4) Independent practice; and (5) Supervisory capacity.

The experts were invited to propose modifications or additional items relevant to hematology residency training EPAs. We sorted, summarized, and statistically analyzed the returned expert opinions of the first round to generate the form for the second round of Delphi consultations, which was based on the results of the first round. Concurrently, experts established the required entrustment level for each EPA across the three-year residency training.

Statistical Analysis

Data were analyzed using SPSS Version 26.0 (IBM Corp., Armonk, NY, USA). Expert engagement was assessed via response rates (>70% required) and authority coefficients (Cr = [judgement coefficient + familiarity coefficient]/2; Cr ≥ 0.7). Consensus metrics included Kendall’s W for inter-rater agreement and CV for score dispersion. Descriptive statistics (mean ± SD, frequencies) summarized EPA ratings and qualitative feedback.

Results

EPA Framework Development

The majority of preliminary EPAs (13/14) achieved overall scores above the EQual rubric’s predefined threshold of 4.07, indicating strong alignment with core EPA domains. EPA 14 (Public health incident response) scored below the threshold (3.23), with all the experts concurring on the need for revision. Following review, the updated EPA 14 (Public health incident response) was ultimately excluded from the training framework. This decision was based on its infrequent occurrence in routine hematology practice, lack of measurable and observable outcomes, and absence of definitive repeatability, all of which fall outside the essential competency requirements for residency training.

Following the initial EQual rubric screening, 13 EPAs proceeded to the Delphi consultations. In Round 1, consensus was not achieved for two items: Identification of HSCT (Hematopoietic Stem Cell Transplantation) indications/contraindications (EPA 7) and Imparting bad news (EPA 13). Ten of 35 experts (28.6%) recommended expanding HSCT competency to include donor selection, while 20/35 (57.1%) proposed removing Imparting Bad News due to its limited hematology-specific relevance and assessment difficulties. These recommendations resulted in revision of EPA 7 and removal of EPA 13. Based on expert feedback, one new item was added to reinforce the 12 items for the second survey round: identification and management of cellular immunotherapy complications, including cytokine release syndrome and neurotoxicity. In Round 2, 13 EPA items were evaluated, with 12 achieving consensus. Only the newly added ‘Management of Cellular Immunotherapy Complications’ was excluded due to insufficient standardization in assessment criteria (Table 1).

Expert Engagement and Consensus Metrics

The Delphi consultations involved 35 experts from eight academic medical centers, representing diverse institutional roles (Table 2). The expert panel demographics showed a median age of 45 years (range: 39–56) and median professional experience of 17 years (mean ± SD: 15.4 ± 6.0 years). Most participants held doctoral degrees (83%, 29/35), with 34.3% (12/35) possessing full senior professional titles and 65.7% (23/35) holding associate senior titles. Participants averaged 11.9 years (SD: 3.6) of experience in standardized residency training programs, providing broad representation of hematology training practices across multiple institutions.

Table 2.

Demographic Characteristics of Expert Panel (n=35)

Characteristic	Value
Institutions (n)	8 academic hospitals
Age (years), median (range)	45 (39–56)
Professional experience (years), median (range)	17 (12–23)
Education, n (%)
Doctoral degree	29 (82.9%)
Master’s degree	6 (17.1%)
Professional title, n (%)
Full senior	12 (34.3%)
Associate senior	23 (65.7%)
Role distribution, n (%)
Residency program director	5 (14.3%)
Clinical department head	2 (5.7%)
Ward director	6 (17.1%)
Frontline clinical teacher	16 (45.7%)
Teaching secretary	3 (8.6%)
Medical Education Administrators	3 (8.6)
Specialty Area, n (%)
Myeloid disorders	10 (28.6%)
Erythroid disorders	5 (14.3%)
Lymphocyte and plasma cell disorders	5 (14.3%)
Hemorrhagic and thrombotic disorders	5 (14.3%)
Hematopoietic Stem Cell Transplantation	4 (11.4%)
Hematologic intensive care	3 (8.6%)
Medical Education	3 (8.6%)
Years in residency training, mean ± SD	11.9 ± 3.6

The recovery rate of every round of expert correspondence questionnaires was 100% (35/35), indicating that the experts had high participation and enthusiasm. The authority coefficient (Cr) for the panel reached 0.85, reflecting high-level expertise and reliability. The Kendall’s W coordination coefficients of the first and second expert consultation rounds were 0.271 (χ ² = 148.894) and 0.529 (χ² = 181.096), respectively. The p value of each round was <0.01, indicating that expert consultation was consistent. Notably, the coefficient of variation decreased from 0.28 to 0.16 for “observability” after revisions, demonstrating enhanced agreement. All items achieved a mean score ≥4.0 and coefficient of variation ≤0.20 in the final round, meeting the predefined consensus threshold. We ultimately developed 12 hematology EPAs, with each EPA comprehensively described along with its applicable clinical settings, potential risks, and operational limitations (Table 3).

Table 3.

List of 12 Entrustable Professional Activities (EPAs) and Description

	EPA Title	Content Description	Potential Risks
EPA 1	Diagnosing and managing common myeloid hematological disorders	Qualification for this EPA includes: Gathering disease-specific information and perform physical examination focusing on hemopathy (especially on the manifestations of anemia, infection and bleeding)	Key information was missing during the collection of medical history and physical examination. The tests were ordered incompletely. Unable to reach a correct and comprehensive diagnosis. Increasing risks, such as misdiagnosis, missed diagnosis, infection, bleeding, organ damage, prolonged hospital stay, and death for patients, were associated with the selection of inappropriate treatment strategies.
		Recommending and interpreting common tests including blood smear/bone marrow tests focusing on myeloid hematological disorders
		Diagnosing and making differential diagnosis of common myeloid hematological disorders (acute myeloid leukemia, chronic myeloid leukemia and myelodysplastic syndrome)
		Developing strategies for the treatment of common myeloid hematological disorder (acute myeloid leukemia, chronic myeloid leukemia and myelodysplastic syndrome)
		Setting: Hematology inpatient ward Limitation: None
EPA 2	Diagnosing and managing common erythroid hematological disorders	Qualification for this EPA includes: Gathering disease-specific information and perform physical examination focusing on hemopathy (especially on the manifestations of anemia, hepatomegaly and splenomegaly)
		Recommending and interpreting common tests including blood smear/bone marrow tests focusing on erythroid hematological disorders
		Diagnosing common erythroid hematological disorders (iron deficiency anemia, megaloblastic anemia, aplastic anemia and hemolytic anemia)
		Developing strategies for the treatment of common erythroid hematological disorders (iron deficiency anemia, megaloblastic anemia, aplastic anemia and hemolytic anemia)
		Setting: Hematology inpatient ward Limitation: None
EPA 3	Diagnosing and managing common lymphocyte and plasma cell disorders	Qualification for this EPA includes: Gathering disease-specific information and performing physical examination focusing on hemopathy (especially on the manifestations of hepatomegaly, splenomegaly and lymph node enlargement)
		Recommending and interpreting common tests including blood smear/bone marrow tests and specific imaging tests (CT, MRI, PET-CT and ultrasound)
		Diagnosing common lymphocyte and plasma cell disorders (acute lymphocytic leukemia, chronic lymphocytic leukemia, multiple myeloma, Hodgkin lymphoma and non-Hodgkin lymphoma)
		Developing strategies for the treatment of common lymphocyte and plasma cell disorders (acute lymphocytic leukemia, chronic lymphocytic leukemia, multiple myeloma, Hodgkin lymphoma and non-Hodgkin lymphoma)
		Setting: Hematology inpatient ward Limitation: None
EPA 4	Diagnosing and manageing common hemorrhagic and thrombotic disorders	Qualification for this EPA includes: Gathering disease-specific information and performing physical examination focusing on hemopathy (especially on the manifestations of bleeding and thrombosis)
		Recommending and interpreting common tests including blood smear/bone marrow tests and imaging tests (vascular ultrasound)
		Diagnosing common hemorrhagic and thrombotic disorders (immune thrombcytopenia, allergic purpura and congenital deficiency of coagulation factors)
		Developing strategies for the treatment of common hemorrhagic and thrombotic disorders (immune thrombcytopenia, allergic purpura and congenital deficiency of coagulation factors)
		Setting: Hematology inpatient ward Limitation: excludes chemotherapy-induced thrombocytopenia (CIT)
EPA 5	Diagnosing and managing common myeloproliferative neoplasms	Qualification for this EPA includes: Gathering disease-specific information and performing physical examination focusing on hemopathy (especially on the manifestations of hepatomegaly and splenomegaly)
		Recommending and interpreting common tests including blood smear/bone marrow tests
		Diagnosing common myeloproliferative neoplasms (polycythemia vera, essential thrombocythemia, primary myelofibrosis)
		Developing strategies for the treatment of myeloproliferative neoplasms (polycythemia vera, essential thrombocythemia, primary myelofibrosis)
		Setting: Hematology inpatient ward Limitation: None
EPA 6	Making differential diagnosis of cytopenia resulting from non-hematological diseases	Qualification for this EPA includes: Gathering disease-specific information and perform physical examination focusing on hemopathy (especially on the manifestations of anemia, bleeding, infection, rash, edema, goiter, alopecia, joint deformity, and lymph node enlargement.)
		Recommending and interpreting common tests to screen for infections, rheumatic diseases, thyroid disorders, tumors and other disorders
		Identifying the indications for blood transfusion
		Developing preliminary strategies for the treatment of cytopenia and consult relevant specialists
		Setting: Outpatient clinic or hematology ward Limitation: None
EPA 7	Assessing the indications and contraindications for Hematopoietic Stem Cell Transplantation (HSCT)	Qualification for this EPA includes: Assessing the indications/contraindications for HSCT (auto-HSCT and allo-HSCT)	It increases the psychological burden on patients and delays treatment.
		Selecting suitable donors
		Setting: Outpatient clinic or hematology ward Limitation: None
EPA 8	Recognizing and managing chemotherapy-related toxicities	Qualification for this EPA includes: Recognizing chemotherapy-related toxicities (eg, nausea/vomiting, mucositis, myelosuppression and organ-specific toxicities)	Failure to identify clinical problems led to delays in treatment and exacerbated the condition. The inappropriate treatment strategies increased risks for patients.
		Developing evidence-based strategies for the treatment of chemotherapy-related toxicities including appropriate dose adjustments and refering to subspecialists
		Setting: Hematology inpatient ward Limitation: None
EPA 9	Recognizing and managing febrile neutropenia	Qualification for this EPA includes: Recognizing febrile neutropenia
		Developing and implementing treatment plans for febrile neutropenia, including timely initiation of guideline-concordant antibiotics, infection source evaluation, and refering to specialists for complicated cases
		Setting: Hematology inpatient ward Limitation: None
EPA 10	Recognizing and managing blood transfusion reactions	Qualification for this EPA includes: Recognizing blood transfusion reactions (eg, allergic reactions, laryngeal edema, anaphylactic shock, hemolytic reactions, and transfusion-associated circulatory overload)
		Initiating evidence-based treatment for blood transfusion reactions and refering to specialists as needed
		Setting: Hematology inpatient ward Limitation: None
EPA 11	Recognizing and managing bleeding	Qualification for this EPA includes: Recognizing bleeding (eg, cutaneous and mucosal bleeding, menorrhagia, visceral bleeding)
		Initiating appropriate initial management (eg, hemostasis, hemodynamic stabilization, referral to specialists)
		Setting: Hematology inpatient ward Limitation: None
EPA 12	Performing procedures for the evaluation of hematological disorders	Qualification for this EPA includes: Identifying of the indications and contraindications for the procedures (making peripheral blood smear, bone marrow aspiration/biopsy, lumbar puncture and intrathecal injection)	It may increase the risk of infection, bleeding, and examination failure in patients.
		Performing technical procedures for the evaluation of hematological disorders (making peripheral blood smear, bone marrow aspiration/biopsy, lumbar puncture and intrathecal injection)
		Setting: Hematology inpatient ward Limitation: None

Entrustment Supervision Levels

Following the two rounds of Delphi consultations, the expert panel established the expected entrustment supervision levels for each training stage based on the EPA framework and Dreyfus model (Table 4). All graduation-level EPA indicators demonstrated CV values below 0.25 for their expected supervision levels, confirming strong consensus.

Table 4.

Expected Levels of Entrustment-Supervision Scales for Residency Trainees in Hematology

EPA	Title	Baseline	Target
EPA 1	Diagnosing and managing common myeloid hematological disorders	2a	4
EPA 2	Diagnosing and managing common erythroid hematological disorders	2a	4
EPA 3	Diagnosing and managing common lymphocyte and plasma cell disorders	2a	3c
EPA 4	Diagnosing and managing common hemorrhagic and thrombotic disorders	2a	3c
EPA 5	Diagnosing and managing common myeloproliferative neoplasms	2a	4
EPA 6	Making differential diagnosis of cytopenia resulting from non-hematological diseases	2a	3c
EPA 7	Assessing the indications and contraindications for Hematopoietic Stem Cell Transplantation (HSCT)	2a	3c
EPA 8	Recognizing and managing chemotherapy-related toxicities	2a	4
EPA 9	Recognizing and managing febrile neutropenia	2a	4
EPA 10	Recognizing and managing blood transfusion reactions	2a	4
EPA 11	Recognizing and managing bleeding	2a	4
EPA 12	Performing procedures for the evaluation of hematological disorders	2a	5

Discussion

The establishment of hematology-specific EPAs through a Delphi consensus by 35 experts represents a critical advancement in aligning CBME with residency training requirements in China. This study addresses the urgent need for context-specific assessment tools within China’s STSR program while contributing to global CBME implementation in hematology and other specialties.

EPAs have been widely adopted and extensively integrated into CBME systems across Western countries.^20,21 In contrast, China’s EPA initiatives remain at an early stage, primarily concentrated on general residency training programs rather than specialty-specific frameworks.²² This gap is particularly pronounced in hematology education, where existing EPA models developed mainly within Western Hematology/Oncology training systems, disproportionately emphasize malignant disorders while inadequately addressing core areas including benign hematologic diseases, transfusion medicine, and coagulation disorders.²³ Such imbalance results in significant training deficiencies for comprehensive hematology practice, highlighting the necessity for specialized EPA frameworks. Our framework adopts a disease category-based approach to systematically delineate clinical practice into distinct professional activities. These essential advancements have correspondingly influenced training requirements.^24–26 This study addresses a critical need by delineating hematology-specific EPAs that encompass the specialty’s dual requirements: proficiency in procedural skills, such as performing diagnostic procedures for hematologic disorders (EPA 12), and sophisticated clinical decision-making, including differential diagnosis of cytopenia with non-hematologic causes (EPA 6). Importantly, these EPAs were specifically tailored to China’s clinical environment, where high patient volumes and resource limitations demand efficient yet rigorous training approaches. Representative examples include EPAs 8 through 10, which focus on recognizing and managing common hematologic conditions. The framework emphasizes both technical skill acquisition and multidisciplinary team coordination, reflecting the essential collaborative nature of modern healthcare delivery.

The development of EPA frameworks generally follows three conceptual orientations: clinical procedure, disease taxonomy, and healthcare delivery systems.²⁷ Firstly, clinical procedure-based EPAs emphasize competencies within diagnostic and therapeutic processes, as implemented in U.S.²⁸ and Chinese²² residency programs (eg, gathering history, performing physical examinations) and hematology-specific EPAs²³ (eg, prescribing chemotherapy regimens, conducting bone marrow biopsies). Secondly, disease category-based EPAs prioritize pathophysiological mastery, illustrated by gastroenterology frameworks²⁹ (eg, managing common acid peptic–related problems) and orthopedics frameworks³⁰ (eg, managing open fracture). Thirdly, healthcare service-based EPAs prioritize population health and systems-based practice, demonstrated in pediatric training programs³¹ (eg, applying public health principles and QI methods). Notably, the integration of these distinct conceptual approaches during EPA development should be approached with caution, to avoid logical inconsistencies or competency framework redundancies. For example, we excluded ‘Imparting bad news’ as a standalone EPA (despite its clinical importance), as 57% (20/35) of Delphi experts opposed mixing clinical procedure and disease taxonomy orientations. Furthermore, the skill remains fully integrated within our taxonomy-based workflow as an embedded competency, particularly in disease-specific EPAs addressing serious diagnoses (eg, leukemia), while intentionally not represented as a discrete item to uphold the framework’s methodological integrity. Beyond establishing EPA titles and descriptions, our framework further specifies applicable clinical settings, limitations and risk factors, providing greater detail than most existing studies and aligning with ten Cate’s EPA design principles.³² Our study developed hematology-specific EPAs based on China’s residency training requirements with a focus on core disease competencies, our rigorous methodology enhances the practical applicability of these hematology EPAs in real-world training environments.

The EQual rubric is a validated tool for assessing EPA quality, enabling developers and educators to identify EPAs requiring substantial revisions through systematic evaluation.¹⁶ When utilized by clinician-educators, it generates actionable feedback regarding structural strengths (eg, clarity of entrustment levels) and content weaknesses (eg, misalignment with clinical priorities), ensuring EPAs meet both educational and practice standards. The integration of quality assurance tools like the EQual rubric is critical not only as a final validation step but also as a prerequisite for Delphi consensus processes.³³ Therefore, our study’s implementation of the EQual rubric prior to initiating the Delphi consultations reduced potential structural and content errors in the EPAs, ultimately improving the efficiency of Delphi consultations and yielding more cohesive consensus outcomes.

To ensure scientific rigor and reliability, we implemented multiple methodological safeguards. We utilized a Delphi method, which allowed researchers to gather and achieve consensus on the opinions of experts through a series of structured questionnaires, which are conducted anonymously to avoid the influence of authority among the experts.³⁴ Furthermore, the expert panel comprised 35 clinical educators and hematology subspecialists recruited from eight academic medical centers across China, including four national-level and two provincial-level hematology referral centers. This selection strategy ensured broad representation across institutional tiers and hematology subspecialties, including benign hematologic disorders, coagulation abnormalities, and malignant hematologic conditions. Additionally, the experts held graduate degrees and senior titles, with substantial experience in residency training, teaching, and management. All completed EPA framework training prior to participation, which supported consistent protocol implementation and high engagement levels. The Kendall’s W coefficient of concordance reflected strong expert consensus. Besides, establishing hematology-specific EPAs represents a critical initiative in postgraduate medical education reform, aiming to enhance educational quality assurance, standardize training outcomes and improve patient safety. To achieve these lofty aims, these EPAs must be rigorously defined and feasible for implementation. Five experts evaluated the EPA alignment with core domains using the EQual rubric,¹⁶ with 12 of 14 items (85.7%) requiring no substantive revision.

These hematology-specific EPAs provide a foundational framework for competency assessment framework for residency training without encompassing all hematology competencies. Focused on resident education, the EPAs prioritize common and high-prevalence hematologic disorders while excluding advanced procedures such as HSCT. Aligned with residency milestones, they emphasize essential skill mastery before specialized training. While we anticipate our EPA list will be valuable, it is not intended as definitive or mandatory for all training programs, but rather as a scientific foundation for adaptation. Furthermore, the expected levels of supervision at various training stages are defined, it is acknowledged that the actual time required for individual learners to achieve entrustment may vary. Establishing maximum competency endpoints remains essential to ensure accountability and clear evaluation standards. Pilot tests and the feedback of practical evidence are still required for EPA application.³⁵ As Amiel et al demonstrated through core EPA implementation in US institutions, such validation reveals both strengths and limitations.³⁶ Broader consensus and participation across China may help provide further validation.

Residents require structured practice opportunities and actionable feedback from clinical supervisors to achieve competency milestones, a need addressed by EPA implementation through workplace-based formative assessments. As demonstrated by Rekman et al³⁷ entrustability scales bridge supervisors’ clinical judgments with standardized assessment tools, enhancing evaluation reliability and validity. Technology integration enhances EPA utility by enabling real-time performance tracking and data-driven feedback, as exemplified by Woodworth et al through their digital platform for anesthesiology training.⁸ These solutions are particularly critical in hematology, given high patient volumes and resource constraints that require efficient, technology-enabled training models. Our team is currently developing a local software tool to track resident performance metrics in real time, visualize progression through dynamic curves, and deliver actionable feedback. This tool will support data-driven, personalized learning plans tailored to individual training needs. A comprehensive analysis will be presented in a follow-up study.

We established a consensus set of 12 EPAs representing essential entrustable activities for hematology residents. These EPAs were validated as effectively capturing core resident responsibilities during hematology rotations. This study contributes to CBME by demonstrating how EPA frameworks can reconcile global standards (including EQual criteria) with local training needs, providing an adaptable model for other specialties in resource-diverse settings. Our subsequent research phase will investigate EPA framework implementation through an existing mobile application to assess its feasibility and effectiveness for hematology subspecialty training. Future work must focus on large-scale deployment across complex clinical learning environments.

Funding Statement

This study was funded by General Research Project of Hospital Management Innovation in Jiangsu Province in 2021 (JSYGY-3-2021-171), the Research Program of Standardized Training for Residents of the First Affiliated Hospital of Soochow Univisity (ZPKJ20210204, ZPKJ20210105) and Research Project of Higher Education Reform of Soochow University in 2021 (5731506122).

Disclosure

The authors report no conflicts of interest in this work.