Design and Implementation of Competency Based Postgraduate Medical Education in Otorhinolaryngology: The Pilot Experience in India

Abstract

The worldwide call for a shift towards competency based postgraduate medical education has until recently gone largely unheeded in India, despite the Medical Council of India enshrining the principle in its regulations for postgraduate institutions. This paper details the first concrete attempt at establishing a CBME curriculum in Otorhinolaryngology in India. The design and implementation of the CBME curriculum was carried out in four phases, in a time-bound manner over a period of 6 months. Phase I consisted of an extensive literature review and a clarification of the major objectives of the program. Phase II involved the listing out of 20–30 entrustable professional activities (EPAs) for each specialty and the 13 core EPAs common to all incoming residents and the subsequent mapping of these EPAs to their respective domains of competence and year-wise levels of competence. This was followed by the development of milestones for each EPA and appropriate clinical vignettes. Phase III focused on development of 360° assessment strategies, including the in-house development of an e-portfolio. Phase IV was dedicated to the implementation of the CBME curriculum, and involved various sensitization and orientation programs for faculty and the new residents. This exercise in designing and implementing a CBME program showed the important role that intra-departmental and inter-institutional cross-communication and exchange of ideas vies-a-vie workshops and personal communication play in bridging the lapses in knowledge in this emerging area, reaching consensus to achieve project goals and for finding relevant solutions to common problems. Medical education in India presents its own peculiar set of logistical and cultural challenges. Keeping in line with the recommendations of the Medical Council of India regarding Postgraduate Medical Education, it is essential that medical colleges in India not fall behind the international paradigm shift towards CBME.

Introduction

All over the world, public demand for accountability is driving a paradigm shift to competency-based medical education (CBME) in the health professions [1]. Medical boards around the world have adopted competency-based frameworks as the underpinnings for new postgraduate training programs. These frameworks include the Outcome Project of the Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialities (ABMS) in the United States [2], the CanMEDS Framework of the Royal College of Physicians and Surgeons of Canada [3], the Scottish Doctor Project in Scotland [4] and the Framework for Undergraduate Medical Education in the Netherlands [5].

The Medical Council of India states in its general conditions to be observed by postgraduate institutions that the “postgraduate curriculum shall be competency based” [6]. Inspite of this core principle being enshrined in the council’s Postgraduate Medical Education Regulations, so far no concrete steps have been taken in the country to establish and implement a competency based medical education program. In view of this lacunae in medication education in India, the Department of Otorhinolaryngology and Head and Neck Surgery, Mahatma Gandhi Medical College and Research Institute, Sri Balaji Vidyapeeth University, Pondicherry, India, embarked on an ambitious mission to design and implement the first competency based medical education program for Otorhinolaryngology in India in January 2016 as a part of a university-wide effort for innovation in medical education. This paper details the various steps involved in the process, the lessons learnt and provides a roadmap for the way ahead, in order to deal with the peculiar challenges of medical education in Otorhinolaryngology in India.

Historical Aspects of CBME

Postgraduate medical education has a relatively short history of structured training, having evolved from an initial apprenticeship model of intense one-on-one supervision and mentorship, concluding with the individual taking over the practice when deemed adequately prepared by the mentor, or with the completion of a national exam or certification. William Halsted developed the first published training program in the 1890s which served as an important model of training that was rapidly adopted by hospitals throughout the United States and Canada. By the turn of the twentieth century, the Flexner Report brought about a sea-change in medical education systems and to this day continues to cast a long shadow over medical training. Problem-based learning (PBL) was first introduced at McMaster University in Ontario, Canada in the late 1960s, largely through the efforts of Dr. Howard Barrows [7].

Outcomes-based learning was first introduced as an instructional strategy by behavioral psychologists as early as the 1950s, but it was not uniformly adopted until much later when several influential reports called for a transition to competency based training and regulatory bodies such as the ACGME and the RCPSC mandated CBME as a requirement for accreditation [8].

Origin and Rationale of CBME

Traditionally, postgraduate medical education has focused upon defining a period of time over which training takes place and the content delivered during that period, and at the conclusion of training, the trainee was considered to be able to practice medicine. Competency-based education however implies a training process that results in proven competency in certain skills and behaviours required to practice that profession. Each trainee is required to demonstrate the acquisition and application of the required knowledge, skills and behavior in order to complete training [9].

The basic essential elements of CBME consist of functional analysis of the occupational roles, translation of these roles (“competencies”) into outcomes, and assessment of trainees’ progress in these outcomes on the basis of demonstrated performance. Progress is defined solely by the competencies achieved and not the underlying processes or time served in formal educational settings. Assessments are based on a set of clearly defined outcomes so that all parties concerned, including assessors and trainees, can make reasonably objective judgements about whether or not each trainee has achieved them [10].

Present Aspects of CBME

Several countries around the world have presently implemented competency-based curriculums in medical schools successfully. But in India, postgraduate medical education continues to be a strict time-bound exercise, and till recently CBME had not been successfully integrated into the regular curriculum, mainly due to a lack of investment of time and resources into the development of CBME. In January 2016, Mahatma Gandhi Medical College and Research Institute, Sri Balaji Vidyapeeth University, Pondicherry, India initiated the process to design and implement a competency-based postgraduate medical curriculum in a time-bound manner within a period of 6 months, with a stated aim to implement the new curriculum in the academic year 2016–2017. The University decided to build a framework based on the ACGME Outcomes Project, with modifications specific to the needs of postgraduate medical education in India.

It is essential that we first define the major components of CBME so as to have a clear vision of what is required of an institution when designing a CBME program.

• Competency is an observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes. Since competencies are observable, they can be measured and assessed to assure their acquisition [11].

• Competency framework is an organized and structured representation of a set of inter-related and purposeful competencies [12].

• Domains of competence are broad distinguishable areas of competence that in aggregate constitutes a general descriptive framework for a profession [13]. The ACGME/ABMS framework identifies six domains of competence: Patient care (PC), Medical Knowledge (MK), Interpersonal and Communication Skills (ICS), Professionalism (P), Practice-Based Learning and Improvement (PBLI) and Systems-Based Practice (SBP). Each domain includes a set of competencies.

• Competency list is the delineation of the specific competencies within a competency framework [13].

• Competence is the array of abilities (knowledge, skills and attitudes) across multiple domains or aspects of performance in a certain context. Statements about competence require descriptive qualifiers to define the relevant abilities, context and stage of training. Competence is multi-dimensional and dynamic. It changes with time, experience and setting [14].

• Entrustable Professional Activity (EPA) is an essential professional work activity or task for medical practice that requires specialized knowledge and skills, and encompasses multiple competencies. They are “critical activities” in the professional life of physicians that the specialty community agrees must be assessed and approved at some point in the ongoing formation of physicians. EPAs as an assessment methodology allow supervisors to observe the performance of a learner in an authentic environment, executing professional work. Observation of a learner conducting an EPA would enable a supervisor to determine the learner’s ability to perform that activity with decreasing supervision and increasing autonomy [15].

• A milestone is a significant point in a learner’s development demonstrated progressively by a learner during the course of their education [16]. They are stages in the development of specific competencies. It is an observable marker of an individual’s ability along a developmental continuum. Milestones are used for planning and teaching.

Steps Involved in Design and Implementation

In order to implement a competency-based postgraduate medical education curriculum in a time-bound manner, the University planned a number of phases through which the workflow was to progress to fruition.

Phase I

In the initial phase, the objectives of the project was to clarify and frame, which included: to specify at least 20–30 EPAs pertaining to each specialty, to map these EPAs to appropriate competency domains, to develop milestones appropriate to each domain, to identify EPAs that required multi-source feedback (MSF), to identify appropriate sources for MSF and finally, to develop an e-portfolio for formative assessment. Following this an extensive literature search on competency based medical education, entrustable professional activities, milestones, multisource feedback and portfolio, was conducted using standard databases.

Phase II

The Department then identified 33 EPAs for MS (Otorhinolaryngology) (Table 1) and 22 EPAs for DLO and the time for regular assessment was defined. Following this, a workshop was conducted wherein 13 core EPAs applicable for all residents on the first day were defined and added to the list of EPAs following general consensus with all departments (Table 2).

Table 1.

Year-wise entrustable professional activities for MS ENT residents

S. no.	EPA	Competency domains						Level of competency				MSF
		MK	PC	PBLI	SBP	P	ICS	Day 1 of residency	End of 1st year	End of 2nd year	End of 3rd year
1.	Gathering a history and performing physical examination	*	*	*	*	*	*	II	III	IV	V	S, P, PG, I
2.	Prioritizing a differential diagnosis following a clinical encounter	*		*			*	I	II	III	IV	S, PG, I
3.	Recommending and interpreting common diagnostic and screening tests	*	*	*	*	*		II	II	III	IV	S, I
4.	Entering and discussing orders and prescriptions and giving the necessary instructions to the patients	*	*	*	*	*	*	I	II	III	IV	S, P, PG, I
5.	Documenting a clinical encounter in patient records	*	*			*		II	II	III	IV	S, PG, I
6.	Provide an oral presentation of a clinical encounter	*		*			*	I	II	III	IV	S, PG, I
7.	Form clinical questions and retrieve evidence to advance patient care	*		*				I	II	III	IV	S, I
8.	Give or receive a patient handover to transition care responsibility	*	*		*	*	*	I	II	III	IV	S, PG, H, I
9.	Collaborate as a member of an interprofessional team	*				*	*	I	II	III	IV	S, PG, H, P, I
10.	Recognize a patient requiring urgent or emergency care and initiate evaluation and management	*	*	*	*	*	*	I	II	III	IV	S, PG, H, P, I
11.	Obtain informed consent for tests and/or procedures	*	*		*	*	*	I	II	III	IV	S, P, PG
12.	Performing general procedures of a physician	*	*	*	*	*	*	I	II	III	IV	S, PG, I
13.	Identify system failures and contribute to a culture of safety and improvement	*	*	*	*	*	*	I	II	III	IV	S, PG, I
14.	Management of dizzy patient	*	*	*	*		*	I	II	III	IV	S, PG, P, I, H
15.	Management of epistaxis	*	*	*			*	I	II	III	IV	S, PG, P, I, H
16.	Management of patient with stridor	*	*	*	*		*	I	II	III	IV	S, PG, P, I, H
17.	Management of foreign bodies of aerodigestive tract	*	*	*			*	I	II	III	IV	S, PG, P, I, H
18.	Preoperative evaluation and counselling	*	*			*	*	I	II	III	IV	S, PG, P, I, H, UG
19.	Post-operative care	*	*
20.	Performing anterior and posterior nasal packing	*	*			*	*	I	II	III	IV	S, PG, P, I, H, UG
21.	Removal of foreign bodies of ear and nose	*	*			*	*	I	II	III	IV	S, PG, P, I, H, UG
22.	Emergency reduction of fracture nasal bone	*	*			*	*	I	II	III	IV	S, PG, P, I, H, UG
23.	Performing adenotonsillectomy operation	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
24.	Performing septoplasty operation	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
25.	Performing middle meatal antrostomy	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
26.	Perform oesophagoscopy and Foreign body removal	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
27.	Perform bronchoscopy and foreign body removal	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
28.	Perform tracheostomy	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
29.	Performing cortical mastoidectomy	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
30.	Performing myringoplasty	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
31.	Performing anterior and posterior ethmoidectomy, sphenoidotomy and frontal sinusotomy	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
32.	Performing head and neck surgeries	*	*			*		I	I	I	II	S, PG, P, I, H, UG
33.	Performing modified radical mastoidectomy	*	*			*		I	II	III	IV	S, PG, P, I, H, UG
34.	Cadaver temporal bone dissection: cortical mastoidectomy, modified radical mastoidectomy	*		*				I	II	III	IV	S, PG, I
35.	Performing common office procedures: indirect laryngoscopy, videolaryngoscopy, postnasal examination and diagnostic nasal endoscopy	*	*	*		*	*	I	II	III	IV	S, PG, P, I, H, UG
36.	Cadaver temporal bone dissection: Facial nerve decompression, Stapedectomy	*		*				I	II	III	IV	S, PG, I
37.	Reading and interpreting all basic ENT X-rays and chest X-ray	*	*				*	I	II	III	IV	S, PG, I
38.	Reading and interpreting PTA and impedance audiograms	*	*		*	*	*	I	II	III	IV	S, PG, I
39.	Reading and interpretation of CT PNS, HRCT temporal bone and CT Neck	*	*			*	*	I	II	III	IV	S, PG, I
40.	Research methodology and writing up a research paper	*				*	*	I	II	III	IV	S, I
41.	Day-care and minor procedures in ENT E.g.: lobuloplasty, keloid excision etc.	*	*			*	*	I	II	III	IV	S, PG, P, I, H, UG
42.	Conducting pure tone audiometry and impedance audiometry	*	*			*	*	I	II	III	IV	S, PG, P, I
43.	Cadaver temporal bone dissection: advanced temporal bone dissection	*		*				I	II	III	IV	S, PG, I
44.	Reading and interpretation of CECT neck, CT cerebellopontine angle, barium swallow, fistulogram	*	*			*	*	I	II	III	IV	S, PG, I
45.	Reading and interpretation of MRI	*	*			*	*	I	I	I	II	S, PG, I
46.	Reading and interpretation of BERA and OAE	*	*			*	*	I	II	III	IV	S, PG, I

Levels of competence

Level I: Knowledge only; can observe, Level II: Can do under strict supervision, Level III: Can do under loose supervision, Level IV: Can do independently, Level V: Has expertise to teach others

Multisource feedback (MSF)

Supervisor: S; Patients/relatives: P; Undergraduate students: UG; Peers: PG; Community: C; Other health professionals: H; Self: I

EPA Entrustable Professional Activity, MK Medical Knowledge, PC Patient Care, PBLI Problem Based Learning and Improvement, SBP Systems Based Practice, P Professionalism, ICS Interpersonal Communication Skills, MSF Multi Source Feedback

* represents tick mark. First EPA has 6 Competency domains, Second EPA has 3 Competency domains and so on

Table 2.

Thirteen core competencies relevant to all residents at the start of the course

Gathering a history and performing physical examination Prioritizing a differential diagnosis following a clinical encounter Recommending and interpreting common diagnostic and screening tests Entering and discussing orders and prescriptions and giving the necessary instructions to the patients Documenting a clinical encounter in patient records Provide an oral presentation of a clinical encounter Form clinical questions and retrieve evidence to advance patient care Give or receive a patient handover to transition care responsibility Collaborate as a member of an interprofessional team Recognize a patient requiring urgent or emergency care and initiate evaluation and management Obtain informed consent for tests and/or procedures Performing general procedures of a physician Identify system failures and contribute to a culture of safety and improvement

Following this, each EPA was mapped to respective and appropriate competency domains and level of competency. The levels of competency were defined as: Level I: Knowledge only; can observe, Level II: Can do under strict supervision, Level III: Can do under loose supervision, Level IV: Can do independently and Level V: Has expertise to teach others. Milestones were then developed for each EPA according to the respective competency domains (Table 3).

Table 3.

Evaluation sheet for postgraduate case presentation

S. no.	Criteria to be assessed	Score
		Below par (1)	At par (2)	Above par (3)
1	Logical order in presentation (History taking)
2	Cogency of presentation
3	Accuracy and completeness of general and local physical examination
4	Other systemic examination
5	Summarizes the case and analyses the appropriate differential diagnoses
6	Whether the diagnosis follows logically from history and findings
7	Investigations required: completeness of list, relevant order, interpretation of investigations
8	Management principles and details
9	Time management
10	Overall performance—relevant answers to questions, attitude during presentation and confidence
		Total score:
	General comments:
	Highlights in performance (strengths)
	Possible suggested areas for improvement (weakness)
	Signature:

Phase III

This phase focussed on assessment strategies for the competency framework devised in the previous phase. A second inter-departmental workshop was organized to discuss the various approaches for collection of MSF and to select the most appropriate one. Also, the need and role of an e-portfolio for formative assessment was discussed and general consensus was reached on the basic design. Appropriate MSF sources were identified for each EPA and mapped accordingly, with the underlying principle that there should be 360° feedback. Various MSF evaluation forms for various activities (including competency-based skills, case presentation, journal club, seminars) and the various stakeholders (faculty, peers, other health professionals, patients, self) were developed and vetted until a general consensus was reached. Next, the University’s Department of Information Technology developed an in-house Learning Management System called Ganesha’s Canvas (elearn.sbvu.ac.in) which is a sophisticated system built on evidence based pedagogic principles that allows students to follow problem based learning, collaborative learning and continuous evaluation. This e-portfolio system was vetted thoroughly in order to incorporate the basic underlying principles of CBME.

Phase IV

While the last three phases were involved with the design of the competency framework and its assessment strategies, the fourth and final phase was concerned only with implementation of the framework. Towards this end, multiple workshops were convened involving all teaching faculty, sensitizing them to the competency framework and its requirements and also what would be required of the faculty in terms of continuous assessment of their residents and strategies for how they could go about it.

The incoming batch of residents underwent a 3 day intensive structured orientation program which introduced the concept of CBME and its requirements and they were trained in the use of the e-portfolio system. Residents were also asked to grade their level of competency for the 13 core competencies, which was then reviewed by their mentors after 1 month. This phase is now on-going and the results at the end of 1 year will be presented in a subsequent paper.

Results

The CBME program in this university has currently entered into the third year, and has involved a total of 8 postgraduates, 2 in year one of training, 3 in year two and 3 in year three. Based on the methodology described above, they were assessed on a regular basis and the gradual progress achieved through the program can be seen in Fig. 1. The 3 residents in year 3 of training are represented as PGY3A, PGY3B and PGY3C, year 2 residents as PGY2A, PGY2B and PGY2C and year 1 residents as PGY1A and PGY1B respectively. As can be seen, the average progress of the residents was tracked through quarterly assessments and appropriate corrective measures to improve performance were applied much earlier in the program, thus ensuring that the residents achieved almost similar competency levels by the end of each year. This helped in standardizing the desired outcomes of the course and ensured that no resident was left behind.

Fig. 1.

Average level of competency of postgraduates assessed every quarter

Advantages and Disadvantages of CBME

There are multiple driving forces for the widespread adoption of CBME. As an instructional strategy, competency based training is based on sound pedagogy which promotes learner-centeredness. CBME also provides a framework for curricula and assessments, as well as flexibility and an opportunity for self-directed learning [17]. The Flexner centenary report suggests implementing defined outcomes and identifying minimum competencies for graduates as a means to promote transparency and accountability of the medical education system and alleviate patient safety concerns [18]. These benefits are particularly important in countries with developing healthcare systems, who are actively building a health professional workforce [19]. CBME is thought to provide transparent standards and increased public accountability and may have the potential to improve efficiency and reduce costs to health care and educational systems [20]. Some suggest that training programs may even be shortened with competency based approaches [21].

Despite these advantages, there are those critical of the unfettered implementation of CBME, who point out the need to be careful about discarding time-based approaches which have served many, and specific worries include the deconstruction of professional practice, the destabilization of education into customized educational programs, the chaos of individualized training timetables, and the impact on patient care with changes to residents’ service delivery [22].

The major barriers to CBME implementation include: the time- and resource-intensive nature of competence assessment, which requires direct observation by multiple assessors in multiple settings; the need for faculty development in teaching and assessing the competencies; a misalignment between learning environments and learners’ chosen practice environments; the logistical challenges of introducing competency-based advancement into a traditionally time-based system (where advancement is primarily based on satisfactory completion of medical school and prescribed number of years of specialty training); and limited investment in health professions education [1, 23].

Implications

This exercise in designing and implementing a CBME program showed the important role that intra-departmental and inter-institutional cross-communication and exchange of ideas vies-a-vie workshops and personal communication play in bridging the lapses in knowledge in this emerging area, reaching consensus to achieve project goals and for finding relevant solutions to common problems. An exhaustive and in-depth review of the available literature prior to the start of the project helps to clarify and crystallize the goals and objectives of the project. This pilot project is still in its infancy, slowly working out minor hitches in its implementation. Regular weekly or bi-weekly assessment of e-portfolios by mentors help the residents receive continuous appraisal of their performance. The practice of a CBME curriculum highlights the need for enhanced assessment practices and tools. Also, as some authors suggest, a matrix educational model that further retains the professional maturation elements of the apprenticeship model while integrating a competency-based model that includes explicit expectations and assessment yet being cautious and avoiding the risk of deconstruction of practice into ever smaller units of competence or of focusing on only those competencies that are easy to describe and assess, would be more appropriate..

Conclusion

Keeping in line with the recommendations of the Medical Council of India regarding Postgraduate Medical Education, it is essential that medical colleges in India not fall behind the international paradigm shift towards CBME. Medical education in India presents its own peculiar set of logistical and cultural challenges, and the major barrier to implementation of CBME is its integration into the already existing medical education system, which is already stretched thin for personnel and resources. So, implementation of CBME requires first, a motivated management with a clear vision to developing and implementing frameworks which are relevant to the local conditions.