National Exit Test: How Will One Size Fit All?

Abstract

Medical education in India is expected to undergo a significant reform after the introduction of the National Medical Commission Act. Single, nationwide National Exit Test (NEXT) is an essential provision under this act, which will be implemented during the next three years. It aims to bring about uniformity in the minimum standard of final MBBS examination, quality control in the licentiate examination and elimination of multiple entrance examinations for admission in postgraduate courses. Since the NEXT has multiple and varied objectives, we suggest a three-step scheme for the conduct of NEXT. Step I (Part A and B) will be equivalent to present-day first and second professional MBBS examinations and step II will act as the final professional MBBS and licentiate examination. Step III will form the basis for admission to the different PG courses. The written exam of Step I and II will consist of structured long, short and multiple choice type questions, whereas practical or clinical examination will consist of structured instruments with lesser inter-rater variability (viz., OSCE, OSPE, OSLER, etc.). This opinion piece is a result of in-depth discussions among major stakeholders such as MBBS students, resident doctors, and faculty of AIIMS, New Delhi. The suggested three-step plan is probably the most feasible way to ensure that the three modalities (final year MBBS, licentiate examination, and PG entrance competitive) are assessed in a valid, reliable, and acceptable manner. Due consideration is given to the fact that an assessment process has a significant impact on learning and teaching. The government may consider these suggestions while formulating the regulations of the NMC Act.

INTRODUCTION

In order to usher in mega reforms in the field of medical education, the Government of India introduced the National Medical Commission Act (NMC Act), 2019.[1,2] The Act aims to form a medical education and health care system that improves the quality and affordability of medical education and ensures the availability of adequate and high-quality medical professionals in all parts of the country.[1] The government considers the Act as a progressive step to ensure integrity, quality, and affordability in medical education along with providing quality healthcare to common people. However, associations of medical practitioners, medical students, and resident doctors have raised concerns [Box 1] over individual sections of the Act including the provisions of the National Exit Test (NEXT).[2] We present this opinion piece to suggest a scheme for the conduct of NEXT after an in-depth discussion with three major stakeholders, i.e., MBBS students, resident doctors, and faculty of AIIMS, New Delhi.

Box 1.

Concerns regarding NEXT

NEXT AS DESCRIBED IN THE ACT

NEXT is supposed to serve three purposes. First, it will act as a passing examination for final MBBS examination. Second, it will act as a qualifying examination to grant the license to practice modern medicine in India for Indian as well as foreign medical graduates. Third, it will serve as a competitive test that will form the basis for admission to the postgraduate (PG) broad-speciality courses in the medical institutions of India.[1]

The NEXT will be conducted by a “designated authority” notified by the NMC/Central government, which is likely to become operational within three years from the date of commencement of this Act.[3] The nitty-gritty of the NEXT (like the pattern of questions, number of papers, etc.) is not yet described and will be specified by the regulations in the act, which is supposed to be formed in due course of time after due diligence.

The primary intent of NEXT could be to ensure uniformity in the level of training in MBBS course (more so in private medical colleges), quality control for medical graduates from foreign medical colleges intending to practice in India, and abolishing the need to take multiple entrance examination and/or multiple counseling processes for admission in PG courses.

CHALLENGES WITH NEXT: WILL ONE SHOE FIT ALL?

There is a marked difference in the fundamental purpose of evaluation in MBBS examination (passing in nature), licentiate examination (qualifying in nature), and PG entrance examination (competitive in nature). While the purpose of the first two examinations is to assess the minimum level of knowledge and competency, the PG entrance examination aims to discriminate between the skills of any two candidates as even the minimal difference in their ranks (merit number) may lead to differences in the course and/or institute offered to them (or available at their rank). Besides, the stakes involved in these three examinations are also different. The PG entrance examination is one of the most sensitive tests of the country that needs to be conducted with extreme caution. Thus, the methods proposed to be used in evaluation for these three modalities will principally require different types of instruments which should to be valid, reliable, and feasible to test these three modalities individually.[4]

ESTABLISHMENT OF CENTRAL BOARD OF MEDICAL EDUCATION

Section 15.2 of the NMC act mandates the establishment of a “designated body” which will conduct final year MBBS examination (for more than 70,000 students), PG entrance examination (of more than one lakh students), and licentiate examination of thousands of the foreign medical graduates. Such a mammoth task will require a dedicated body. Central Board of Medical Education (CBME) can be the “designated authority” mentioned in the NMC Act. The success of NEXT will entirely depend on the continuous inflow of skillfully constructed questions for written/theory and clinical/practical examinations. Constant research and development will be instrumental in improving the validity and reliability of such a high stakes examination. There is a general perception that PG entrance examination in India (like NEET PG and SS) has poor validity and reliability in comparison to similar examinations from the west like USMLE that tests a candidate over 2-3 days with thousand-odd questions. Establishment of a dedicated central examination body will be an extremely crucial step. Faculty already involved in various examinations in the country (like AIIMS entrance, PG entrance) and qualified in the field of medical education with an inclination toward assessment system may be deputed/attached to this board. CBME can be a massive success in bringing quality improvement in the field of medical education, in the same way as Central Board of Secondary Education has been in the secondary and higher secondary school education in India.

PROBLEMS AND CONCERNS

There are several concerns regarding the scheme of NEXT among major stakeholders [tabulated in Box 1]. Education drives learning and we need to formulate a method directed to meet the healthcare goals of our country without compromising on the quality while looking for the means to achieve quantity (like meeting doctors-patients ratio). While doing so, we also need to devise a strategy that facilitates the engagement of appropriately trained foreign medical graduates. The current method of evaluation (MCI screening test) has already created a backlog of about 2 lakh students who are not able to clear this examination after getting MBBS/equivalent degrees from various countries. Some of them might be competent enough to serve at primary care level (equivalent to proposed community health workers). However, they might be failing multiple times to clear screening test due to deficiencies in our evaluation system (like number, nature, and type of questions asked in the screening examination). A similar situation may arise if NEXT is conducted without due deliberations and preparedness. This may have a negative impact on the psychology of these students, their parents, and society in general. In the initial few years, it would be appropriate to set the level of NEXT examination relatively easier, (with must know questions mainly) to build up the confidence of students, parents, and society.

ASSESSMENT METHODS COMMONLY USED IN MEDICAL EDUCATION

Commonly used tools for written, practical, and clinical examination in medical education are summarized in Tables 1 and 2.[5,6] Readers are advised to go through the tables to understand the basis for our suggestion. It is important to note that all assessment methods have strengths and limitations and no single method (in isolation) can assess all types of knowledge and skills and, thus, a mix of methods depending upon the context is suggested.

Table 1.

Basic description of the written examination assessment methods used in medical education

Methods of assessment/Domains evaluated	Strength	Limitations	Remarks
1. MCQs (Knowledge and problem-solving ability)	A large number of items encompassing many content areas can be evaluated, high reliability, less time consuming, computer can evaluate answer sheet.	Making of good quality MCQs requires lots of time and expertise, can result in Cueing.	Various formats of MCQs like (single and multiple options), extended matching and assertion-reasoning can be useful a large number of samples in a large number of candidates in a short time.
2. SAQ (Problem-solving ability, reasoning skills, interpretation)	Can evaluate reasoning and problem-solving ability. Avoid queuing	More time consuming, Check less number of samples.	The preset marking scheme is essential to reduce subjectivity. Less suitable for broad sampling.
3. LAQ (Ability for information synthesis and interpretation)	Can evaluate high order cognitive process, Avoid cueing.	Time-consuming, inter-rater variability.	The preset marking scheme is vital to reduce subjectivity. Mainly used in preclinical examinations. Unsuitable for PG entrance examination.

MCQ: Multiple choice questions, SAQ: Short answer type questions, MEQ: Modified Essay questions, LAQ: long/essay type questions

Table 2.

Methods for the assessments used in the evaluation of clinical skills in medical education

Methods of evaluation	Description, Strengths, and Weaknesses
Long Case	Assess clinical competency based on the presentation of history and clinical examination followed by an oral discussion on history taking, physical examination, diagnosis, and treatment plan. It presents a complete and realistic clinical challenge for the examinee. However, it has limitations of an unobserved patient encounter, marked inter-rater variability low validity and reliability. Abandoned in countries like North America.
Short Case	Assess clinical competency based on supervised focused physical examination of a real patient. Assessment is done on the student’s examination technique, ability to elicit physical symptoms and interpret findings correctly. No actual patient or direct observation. Unstructured and subjective.
OSLER	Students, after completing the workup like a long case, are assessed on the standard ten items over 20-30 min by the examiner. Four items are based on history taking, three on physical examination, and one on investigations, management, and clinical acumen. Much importance is given to the process of history taking and communication skills. More valid, reliable, and objective than traditional long case evaluation.
mCEX	Under the direct observation of assessor, the student takes a focused history and/or conducts a physical examination and provides diagnosis and treatment plan. Presents realistic case challenge and allows more extensive case sampling. However, it is resource-intensive.
DOPS	Assessment of candidate’s performance under direct observation in performing procedural skills on real case or mannequin under direct supervision. This method has high validity and reliability.
OSCE	Assessment of competency through direct observation of candidates rotating through a sequence of timed stations dealing with standardized patients. Checklist of specified behaviors or global rating form is used to assess for a large number of samples. Valid and reliable tool for assessment for a specific set of skills. However, ideal book-type scenarios may differ from real patients. This method is resource-intensive.

OSLER: Objective Structured Long Examination Record, OSCE: Objective Structured Clinical Examination, mini-CEX: Mini-Clinical Evaluation Exercise, and DOPS: Direct Observation of Procedural Skills

Tool(s) of evaluation should be chosen carefully based on the context and purpose. The consideration of utility criteria of the assessment tool (viz., validity, reliability, educational impact, acceptability, and feasibility) can be helpful in making an appropriate choice. In real life, there is always some swapping between validity and reliability. For high stakes, competitive examinations, more reliable instruments should be used, whereas, for formative assessments, tools with higher educational impact should be chosen.[5]

Assessment methods need careful planning and strategic designing. It should be studied for its educational effects; otherwise, it can promote unhealthy approaches to learning. What is evaluated and which methods are used in the assessment significantly guides what will be learnt by the students.[4,5] This is quite evident in the current scenario, where the MBBS students are found missing from clinical postings and enrolling themselves in coaching classes in the very first year to prepare for MCQs for PG entrance examination.

SUGGESTED SCHEME OF NEXT

The suggested scheme to conduct NEXT is summarized in Table 3. The scheme has been proposed after giving due consideration to the context and purpose of assessment. Besides, the feasibility of selecting a particular tool for the designated purpose has also been considered. There has been increased concern about suboptimal training facilities and infrastructure in many private medical colleges.[7] Step I of the suggested scheme will act as a quality check for such colleges and will motivate them to improve the standard of teaching and training to an acceptable level. Although the act has left the examination of the first and second MBBS examination to the respective colleges/universities, we are of the opinion that the conduct of step I by the CBME will improve proper training in first and second professional subjects that are equally important in medical science. Many countries like USA, UK, Canada, and Australia conduct central licentiate examinations. Steps I and II of the suggested scheme are similar to that of USMLE in terms of the content (medical subjects covered in steps I and II) with a slight change in the types of question.[8] We have suggested the use of structured LAQs and SAQs in steps I and II along with MCQs (USMLE uses MCQs only) for two reasons. First, no single modality of evaluation is ideal, and every instrument of evaluation has its advantages [Table 1 and 2]. Hence, a mixed bag of all types of questions will be most appropriate (valid and reliable) to judge the knowledge and competency of the students. Careful structuring and preset marking scheme will be required to avoid the inter evaluator variability in the marking and evaluation of SAQs and LAQs. Second, the validity of MCQs based examination entirely depends upon the framing of good quality questions which is a resource-intensive process (requires time and skills). Steps I and II conducted by a central university could be the most suitable alternative to the current system of university examinations which has failed to stand up to the expectations in many private medical colleges. Besides, the CMSE will extend infrastructural support to many newly opened/proposed medical colleges. Step II will also involve a clinical evaluation, which would consist of Objective Structured Clinical examination (OSCE), Objective Structured Long Examination Record (OSLER), Mini-Clinical Evaluation Exercise (mini-CEX), and Direct Observation of Procedural Skills (DOPS) that are more reliable tools with less inter-evaluator variability.[9,10] One long case and a short case can also be considered with proper structuring and preset marking scheme. Careful structuring and preset marking scheme will avoid the inter-evaluator variability. STEP-II in the proposed scheme of NEXT will act as the final year MBBS examination as well as licentiate examination for Indian medical graduates. Both the written test and the clinical/practical examination should be conducted by the central board in a non-home center to ensure integrity. The foreign medical graduates can appear in the randomly allotted designated centers for taking licentiate examination.

Table 3.

Basic details of the suggested scheme of the evaluation under the NEXT

Component of NEXT	Pattern of the examination	Importance/Remarks	Challenges/Other points to consider
Step I (Part A and B) Equivalent to present-day First and Second MBBS examination Timing: After one and two and a half years of the admission for part A and B.	Theory: SAQ, MEQ, LAQ, and MCQs Practical OSPE, more of structured instruments	Ensure uniformity in the minimum level of teaching in the various medical colleges, primarily private and newly opened medical colleges.
Step II Equivalent to present-day Final MBBS examination Timing: After four year and half years of the admission	Theory: More of Objective questions: MCQs (Different types). Few structured SAQs and case scenarios based LAQs can be kept preset marking schemeClinical Evaluation: Most of OSCE, DOPS, mini-CEX. Should be conducted in the non-home center.	Final year MBBS passing examination- Will act as a quality indicator of the training and teaching of respective medical college. Licentiate examination: Will define a uniform minimum level of competency in all medical graduates (both Indian and foreign) intending to practice in India.	How the cut off for passing the examination will be determined?
Step III Corresponding to the present day PG entrance examination. Timing: Can be taken anytime after Passing Step II. Multiple attempts should be allowed to improve rank	Objective questions based-different types of MCQs (single and multiple options), extended matching type questions, assertion-reasoning-based objective questions.	Basis for the entrance in PG medical courses	Skillfully constructed good quality questions will be required. How will scores of different sessions be compared in preparing merit-list?

MCQ: Multiple Choice Questions, SAQ: Short Answer-Type Questions, MEQ: Modified Essay Questions, LAQ: Long/Essay Type Questions, OSCE: Objective Structured Clinical Examination, mini-CEX: Mini-Clinical Evaluation Exercise, and DOPS: Direct Observation of Procedural Skills

Step III will act as a competitive entrance test and form the basis for admission to the different PG courses. Approximately, one lakh twenty-five thousand candidates compete for thirty thousand odd seats. Considering the factors of reliability and feasibility, it should be MCQs based. Different formats of MCQs (single and multiple options, extended matching type, assertion-reasoning, etc.) can be used to ensure widespread sampling. The number of items should be adequate to ensure proper sampling of the final year syllabus. The validity and reliability of this test will depend upon skillfully constructed good quality questions. Candidates can take Step III after passing Step II before the internship. Besides, step III can be taken multiple times to improve rank, if desired by the students. Multiple sets of questions used in the different time frame with different difficulty levels (even if notionally) may lead to the controversy. The issue regarding the decision of inter-se merit for counseling can be addressed by validated statistical methods like “normalization procedure adopted on the basis of percentiles.”

SEPARATE BOARD OF EXAMINATION FOR AIIMSS AND OTHER INIS

To create medical excellence in country, visionaries and creators of the medical education system of the country have established All India Institute of Medical Sciences (AIIMS) and other INIs (Institute of National Importance) by a separate Act of parliament. These institutes have proven their high standards in the field of patient care, medical education, research, and training decade after decade. They have the mandate to act as an apex referral center for patient care, conduct cutting edge medical research, and formulate guidelines for different diseases based on the local needs and conditions. This mandate is notably different from other medical colleges in India, which is mainly to take care of basic medical needs of the country, act as tertiary level referral centers for patient care and run various undergraduate and postgraduate courses like MBBS, MD/MS, and DM/MCh. Thus, the content of training in these institutions and the nature of competencies/skills expected to be attained by students in these institutes will be different. Besides, these institutions (AIIMS, New Delhi and PGI Chandigarh) have a different pattern of intake of students (biannual intake) in postgraduate courses from sate medical colleges (that have an annual intake), including their high-quality entrance examinations which are most difficult examinations by any standards. Assessment is a controlling driver of learning, and it gives guidance to learners about what they should be learning.[11] Keeping the same assessment system for the students of these INIs will ultimately lead to deviations in the mandate of these institutes.

Our suggestion to have multiple steps in NEXT and the use of mixed types of questions for assessment by a central board will be crucial in the quality improvement of medical education in our country. The suggested scheme is probably the most feasible, valid, reliable, and acceptable option to achieve the three main intentions of the NMC Act (in medical education), viz., ensuring uniform and quality teaching in medical colleges (via Step I, II, and III), a minimal level of competency to grant the license to practice (for Indian and foreign medical graduates) through step II, and a single and valid PG entrance examination (Step III). Government of India may find these suggestions useful while formulating the regulations of NMC Act 2019.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.