Identifying entrustable professional activities for post-graduation in ENT: What should an ENT specialist be able to do?

Abstract

Background

Though regulatory bodies of medical education in India mandate post graduate medical education to be competency based, the implementation of the same has not kept pace. The present study aims to develop and propose a set of entrustable professional activities for a masters degree in the discipline of ENT. It also attempts to present it in the form of a portfolio.

Methods

Experts in the subject went through a series of activities including brainstorming, discussions and internal peer review to shortlist and enumerate EPAs.

Results

335 EPAs were identified and are presented. They were further classified as per need and expected expertise.

Conclusion

The EPAs may serve as a guideline to medical colleges/institutions aiming to adopt competency based medical education in the post graduate curriculum in the subject of ENT.

Introduction

What makes a MBBS graduate doctor a specialist? Is it working with another specialist for three years? Passing a theory exam? Presenting a case? Many a post graduate teachers would have reflected on this question, especially at the end of a summative post graduate exam. Watching students perform well in the viva-voce and case presentations, the examiners may feel satisfied; but at times a tinge of doubt creeps in - will the resident be able to deliver as an independent entity in the community. Can the resident be trusted to deliver? Is the assessment and certification process then aligned to gaining this trust?

In the context of ENT, regulatory bodies in the country have defined the curriculum and syllabus well and prescribed it to be competency based.^1,2 However, they do not define the desired outcomes in clear specific terms. Like any other outcomes-based education, there need to be clear answers about what should the specialist be able to do? What can he/she be trusted with? And, if so? What is the path to gain that trust?

The clear and explicit enlisting of what a learner should be able to do has been referred to as Entrustable Professional Activity (EPA). These are not theoretical needs for passing an exam but statements of ability for which a theoretical background may be needed. Milestones are referred to as intermediate steps needed to reach a level of trust. This concept has been used since the late 1990s to help draft CBME curricula in many education systems across the world, the most notable being the Outcomes Project of ACGME.³ The advantages of defining EPAs are many and need to be incorporated into the system.⁴ So, though a curriculum for Masters in Surgery (MS) in ENT exists in the country, outcomes in terms of EPAs have not been defined by the regulators. There is a definitive need to identify EPAs and milestones to help the learner reach the competency.

Most post graduate disciplines including ENT, require the learner to acquire higher order skills as they work through the years of post graduation. This on the job learning needs to be documented for validity. Educational portfolios are one such validated tools to document learning during an educational course.⁵ Log books, another popular tool to document learning do exist. However, they lack the opportunity to serve as valuable reflective learning tools. A portfolio provides this opportunity to document and reflect learning experiences.

The present study aims to identify and enumerate EPAs for a learner of Masters in Surgery in the discipline of ENT in the Indian context. They may be further documented in the form of a portfolio along with milestones to aid all stakeholders in the education process to have a clear and explicit understanding.

Material and methods

The present study was done in the Armed Forces Medical College, Pune. This institute has been conducting regular post graduate courses for more than 50 years. The exercise to identify EPAs first included creation and identification of a study group. The study group consisted of 4 professors in the subject and 4 final year residents. The entire subject curriculum was divided into smaller segments for the sake of classification. Following this the study followed four phases to develop the EPAs. The outline of the study methodology is shown in Fig. 1. What constitutes an EPA was as defined by ten Cate in 2005.⁶ The participants were explained the constituents and attributes of an EPA accordingly.

Fig. 1.

Outline of Study Methodology.

Phase of Divergent Thinking

Each member of the group was asked to list out what he felt was needed to perform as an ENT specialist in either of two conditions. These were - working as a standalone ENT specialist in a peripheral setup and secondly working as a team-member in a ENT department of a tertiary care hospital. They were asked to respond using the categories developed and in the form of ‘job requirements’. This activity was done by the participants individually over a month and participants were encouraged to give as many ideas/suggestions as possible. The inputs of the participants were collated and compiled using an online form/database.

Phase of convergent thinking

The group sat together and a group discussion was done to shortlist and remove overlaps in the draft EPAs. During this process, effort was made to critically think and merge and reclassify EPAs. A simple consensus was needed to adopt or drop a statement. This was done in an unblinded fashion in a single room over multiple sessions. Suggestions from other experts (Medical Education Experts/Audiology/Speech Language Pathologists) was sought on an as required basis.

Phase of deconstruction

During this phase, each EPA was discussed in detail. They were classified as ‘Vital’, ‘Essential’ or ‘Desirable’. Additionally, the expected expertise of each EPA was classified as ‘Knows’, ‘Knows How’, ‘Shows’, ‘Shows How’ and ‘Does’ keeping in mind the expectations from a just graduated ENT surgeon. Some EPAs were broken down into smaller tangible and recordable milestones. The milestones were defined as smaller steps which could be observed and assessed during the course of residency. The expected level of expertise needed for each milestone was also identified. This was done by all members of the group except for 02 senior members who participated as internal peer reviewers in the subsequent phase.

Phase of internal peer review and corrections

This was followed by creation of the draft portfolio and peer review by the team members. Each of the two internal peer reviewers went through each EPA separately and suggested changes. These changes were incorporated and a second round of internal review was done. Following two rounds, the portfolio was finalised.

Framework of developing the portfolio

The template developed by Datta et al was used as a framework to populate the EPAs and milestones.⁷ This template has been especially designed to create a portfolio to meet needs of outcomes based curricula and has EPAs and milestones as the primary ingredients. All compilation of content and development of the portfolio was done using a shared electronic spreadsheet programme (Google Sheets). To create the individual pages of the portfolio, an open source mail merge tool was used. Only the peer review process was done using physical copies to enable a more diligent scrutiny.

Results

The study was conducted from Apr–Nov 2020. The study group as already mentioned in the methods, initially identified the broad sections and segments for sake of classification from the syllabus. The group identified 07 broad groups and 35 segments within this grouping. The classification is as shown in Table 1.

Table 1.

Sub Classification of the speciality.

S No	Section	Segments
1.1	General ENT	General ENT
1.2		OPD skills
1.3		Minor OT Skills
1.4		ENT Emergencies
1.5		Research & Publications
1.6		Promotive/Community ENT
2.1	Ear	General Otology
2.2		Facial Nerve Disorders
2.3		Balance disorders
2.4		Hearing Rehab
2.5		Lateral Skull Base
2.6		Operative Otology
3.1	Nose	General Rhinology
3.2		Nasopharyngeal lesions
3.3		Rhinoplasty & Facial Plastics
3.4		Sleep Disordered Breathing
3.5		Anterior skull base
3.6		Maxillofacial Trauma
3.7		Operative Rhinology
4.1	Throat	Oral Cavity and throat
4.2		Laryngology
4.3		Tracheobronchial lesions
4.4		Esophagology
4.5		Operative Laryngology/Upper Airway
5.1	Head Neck Surgery	General Head Neck
5.2		Thyroid/Parathyroid
5.3		Salivary Glands
5.4		Head Neck Oncology
5.5		Reconstructive surgery
5.6		Operative Head Neck
6.1	Audiology & Speech Pathology	Audiology
6.2		Speech Language Pathology
7.1	Administrative ENT	Medical Stores
7.2		Training
7.3		General Management

During the first phase (Divergent Thinking), the group proposed a total of 1167 EPA statements were generated. After merging/removing duplicates, 639 statements were identified. The group discussions helped to narrow down the EPAs to 401 by merging and reassignment. This involved removing skills that could be acquired as part of another EPA. At times a separate EPA was added to make the learning outcomes more explicit. The group then subdivided the work and classified the EPAs as per expected expertise and level. For EPAs where milestones were defined, an effort was made to make the milestones as measurable activities, either during formative assessment or amenable to workplace assessment. The expected expertise needed for each milestone was also stated alongwith. The draft template prepared went through 2 cycles of peer review and corrections as discussed in the methods. It was felt that the milestones should not be too prescriptive and some flexibility incorporated in the system to cater to local adaptations.

In the final portfolio, 335 EPAs were incorporated. The final list of all EPAs is being presented as Supplementary material to this article. A sample portfolio page with milestones as per the template page is shown as Table 2. The break up of the EPAs from various sections is depicted in Fig. 2. Out of the total EPAs, 58.2% were considered Vital, 19.7% as Essential and 22.1% and Desirable (Fig. 3).

Table 2.

Sample portfolio page showing various elements pertaining to a single EPA.

EPA No: 26	Segment: OPD skills	To be completed by:
EPA: Ability to perform flexible fibre optic laryngoscopy
Expected level: Vital	Expected Domain expertise: Does
Suggested Teaching Learning Experiences: Skills Lab, Apprenticeship
Suggested Assessment Methods: OSCE, WPBA

S no	Milestone (Instructor to initial with date in the box once he/she is satisfied with the level achieved)	Expected Expertise	Endorsed By	Date	Evidence (if any)	Remarks
1	Demonstrates knowledge of endoscopic upper airway anatomy	Knows
2	Demonstrates knowledge of indications/contraindications of the procedure	Knows
3	Performs satisfactory pre-procedure preparation of the procedure including equipment setup	Does
4	Counsels the patient/attendant about the procedure	Does
5	Applies appropriate and adequate anaesthesia for the procedure	Does
6	Demonstrates good visualisation of endoscopic anatomy and hand eye coordination	Does
7	Documents and interprets the findings with reasonable accuracy	Does
8	Performs the procedure consistently with adequate patient comfort and safety	Does

Reflections of student (What have you learnt and its implications on your professional development).

Feedback by facilitator.

Fig. 2.

Section-wise EPAs.

Fig. 3.

VED analysis of EPAs.

Discussion

Defining the outcome of an educational process is a fundamental part of competency based training. In the Indian context, though regulators have mandated a CBME curriculum and provided guidance, it has left it to the colleges and universities to formulate the curriculum delivery and assessment. To fully appreciate the benefit of CBME, it is important to incorporate learning at the workplace. Though such learning already takes place in most PG institutes and colleges, there is a lack of structure and learners are often learning in an opportunistic manner. The explicit declaration of what is expected of the learner will help all stakeholders especially the students to have a clear understanding of what is expected. They will also be under some pressure to acquire all vital EPAs and will give them a good overall exposure within the speciality. Though CBME curricula are student driven, teachers have a great responsibility to provide and mould the teaching learning environment to help students acquire EPAs. They are closely observing them at the workplace and need to update and adapt their teaching learning to the workplace. The use of a portfolio is this context will add value and encourage the learner to be self directed and self motivated. Reflections in the portfolio will help the learner internalise the lessons gained during training.

The need for defining EPAs and milestones has been felt by the medical education community in the country.⁴ However, only a limited number of EPAs have been identified and enlisted for post graduate courses. One such attempt in pathology used a survey and questionnaire method.⁸ In ENT as well, 33 EPAs were identified apart from 13 core EPAs in the subject by Karthikeyan et al.⁹ However, we feel that limiting the EPAs to 33 may be too simplistic and may not be comprehensive. The present set of EPAs developed by us represent a more detailed and specific set of desired outcomes. The present study also defines milestones in the form of clear statements. Some of the EPAs defined in the Karthikeyan et al, are based on cadaver dissection which we believe is not an outcome but a method to reach an outcome. However their efforts are appreciable and perhaps more colleges need to attempt and report the incorporation of EPAs in CBME curricula.

The present set of EPAs are only the first iteration and will need more changes as they are adopted. They may be taken as a set of job descriptions of an ENT surgeon after finishing a Masters degree. This may also vary from place to place based on institutional requirements. Similarly, the milestones may vary depending on local circumstances and should be developed by the implementing institutes. We have provided a sample page of a portfolio based on a single EPA to serve as a guide (Table 2). We believe the present set of EPAs represents a fairly accurate and complete set which can be adopted by any institution conducting the course and wanting to adopt CBME. The enumeration of milestones for each EPA needs special mention. Each EPA needs to be further deconstructed into discrete milestones as has been shown in the example (Table 2). As already mentioned, this needs to be less prescriptive and medical educators need to adapt them to their needs/feasibility.

The implications of developing these EPAs are important. They have the potential to be used as a portfolio as proposed. However, even otherwise, they become a roadmap for the learner, teacher and management in providing fixed clear outcomes. It is likely that some EPAs are not as important or vital as others. The addition of a VED scale is important for this purpose. Though by definition, EPAs are essential, we have listed EPAs which may not be vital but desirable to allow flexibility to institutes where better facilities are available. However, it is important that students do not spend too much time in EPAs that are desirable at the expense of Vital. In tertiary care referral teaching hospitals, there is a definitive risk of focussing too much on some EPAs at the expense of others. This tendency needs to be avoided so that the ENT surgeon is competent in all subspecialties.

Workplace based learning and assessment are fundamental to the concept of a CBME curriculum. This is more so in a post graduate course. We have therefore adopted the use of a portfolio model using milestones. Also, to cater to the local needs, teaching learning activities and assessment methods have been suggested for adaptation. Teachers adopting the portfolio can tweak these methods to suit local conditions. The portfolio also allows the teacher to log the progress and provide feedback to the student during the learning process.

Medical educators in the speciality can also identify gaps in the teaching learning experiences in the curriculum. This may help them focus on improving infrastructure, resources or cater to providing alternate teaching exposures in the form of rotations and cross attachments. An example might be EPAs related to maxillofacial surgery, rhinoplasty, oncology etc where gaps can be identified in the portfolio and timely intervention done.

Limitations

This represents an initial attempt to elucidate the outcomes of a MS (ENT) course in terms of EPAs. A word of caution is however in order. Though the EPAs have been enlisted and internally reviewed, they have not been tested. Testing an educational tool requires a long follow-up, especially to study the change in behaviour of the student (Kirkpatrick Level III) and beyond. Also, the list has not been reviewed by experts from external agencies and limited to one institution. This may allow a bias to creep in the list. We do however think this provides a sound and fundamental foundation. Perhaps identification is a first step and we look forward to colleges incorporating them in their curricula and report results. This is just the creation of a portfolio and it does not represent any education intervention as yet. We also intend to create an electronic version of the portfolio with separate milestones for each EPA so that it can be suitably incorporated.

The recent publication of AMEE Guide No. 140 recommends the way an EPA should be elucidated into eight sections.¹⁰ The present listing of EPAs in this study does not follow this guideline. It is possible to adapt the EPAs into a detailed breakdown as recommended in the guide. The present objective of the study was however, to focus more on developing a comprehensive list of EPAs.

Conclusion

Competency based post graduate education is a mandated and essential pedagogy in medical education. An explicit statement of what the finished product of an MS (ENT) intensive three year residential course is supposed to do in the form of EPAs is lacking in the country. The current article attempts to spell this out in a clear manner. Incorporating the same using an educational portfolio consisting of EPAs and milestones has the potential to catalyse the implementation of PG - CBME in the discipline. Though this is just a beginning, it is hoped that medical colleges adopt the same to generate the data and identify challenges to implementation.

Disclosure of competing interest

The authors have none to declare.

Appendix A. Supplementary data

The following is the Supplementary data to this article: