Abstract
BACKGROUND:
In curriculum reforms under Competency-Based Medical Education (CBME) introduced by the National Medical Commission, medical students are recognized as one of the stake holders. The well-designed multiple-choice questions (MCQs) help students in assessment as well as promote deep learning. The objective of this study is to assess the perceptions of the Indian medical students towards the examination patterns in their undergraduate curriculum with particular reference to MCQs.
MATERIALS AND METHODS:
An observational cross-sectional study was conducted among the medical colleges in India. A total of 7875 students took part in the study. The study instrument was a validated pretested self-administered questionnaire. The study was conducted as an online survey using the Google Forms platform. Inclusion criteria were students from all professional years as well as interns and those who were willing to participate in the study.
RESULTS:
For the question of initiation of MCQs as a tool for the assessment, many of the students (46%) expressed that it should start from the 1st MBBS itself. The majority of the students (85%) felt that MCQ oriented preparation will be helpful for MBBS professional (theory and practical) examinations. Most of the students (90.9%) preferred that the MCQs should be concept and case scenario based. To the question of whether the current system of the university theory examinations of essays and short notes be replaced in total with MCQs so that they are more prepared for NEXT and USMLE examination, 65.5% responded with an answer of yes.
CONCLUSION:
The findings of this study provide input to the regulatory authorities and other stakeholders responsible for designing and implementing the medical curriculum.
Keywords: Assessment, India, medical students, multiple-choice questions
Introduction
The National Medical Commission (NMC) has been constituted by an act of Parliament known as the National Medical Commission Act, 2019, which has come into force in place of the Medical Council of India with the aim to improve access to quality and affordable medical education.[1] Currently, the doctor to the population ratio in India is about 0.77: 1000 as per the current population estimate of 1.33 billion against the WHO norms of 1 doctor for 1000 population.[2] There are more than 500 medical colleges in the country, and on average 82,950 students graduate from the medical schools annually.[3]
However, the skill set of these students has been questioned and to set this right, the erstwhile Board of governors in supersession of the MCI has introduced the Competency-Based Medical Education (CBME) from the year 2019. The CBME focuses on the development of competencies designed to train the Indian medical graduate to be a physician of the first contact within the health system.
The Miller's Pyramid depicts the various levels of acquiring clinical competencies starting with the knowledge to actually acquiring the skill. Hence, the foundation of a good knowledge base cannot be overemphasized. The assessment of knowledge is by means of various methods such as essays, short answers, and very short answers, and multiple choice questions (MCQs). The Bloom's taxonomy provides the taxonomic levels from knowledge to synthesis in increasing levels of complexity. However, there has been a criticism that the assessment of the students knowledge is usually at the level of fact gathering. The testing of recall or comprehension is through essays and short essays. The higher levels of knowledge acquisition, i.e., interpretation and application can be assessed by methods such as case presentations, essays, and extended matching type MCQs. The MCQs especially well-designed help students to be assessed for application which in turn promoted deep learning. Hence, MCQs have become an important assessment method used in the screening tests for medical students to get admission to postgraduate studies.
In curriculum reforms under CBME, the medical students are recognized as one of the important stakeholders. However, there is limited literature about the perceptions of medical students about the examination pattern and the use of MCQs as a method of assessment. The aim of the study is to promote the use of MCQs in the assessment of students at the undergraduate level. The objective of this study is to assess the perceptions of Indian medical students towards the examination patterns in their undergraduate curriculum with special reference to MCQs.
Materials and Methods
Study design and setting
An observational cross-sectional study was conducted among the medical colleges in India. The study participants were undergraduate medical students. The study was conducted with data collection in November 2020. Inclusion criteria were students from 1st, 2nd and 3rd MBBS Part I and Part II as well as interns and those who were willing to participate in the study.
Study participants and sampling
Exclusion criteria were students who have completed an internship or those who were transferred in from medical school from outside India. The minimal sample size obtained is 5644 using formula 4pq/l2 (taking proportion [p] of students who preferred MCQs [83%] from the pilot study conducted on 1234 students, absolute error [l] of 1%, at 5% alpha error, 80% power). The sampling technique used was convenience sampling. A total of 7875 students took part in the study. The study instrument was a validated pretested self-administered questionnaire. The study variables included background information such as name and year of study, type of institutes such as government or private, gender, state, and perception about the current examination pattern. Institutional ethics committee clearance was obtained for conducting the study. The confidentiality of the participants was maintained.
Data collection tool and technique
The study was conducted as an online survey using the Google platform. The questionnaire was designed as a Google Form and sent as an online survey form across all the colleges across the country. Two student research organizations, namely, Andhra Medical College Undergraduate Research Association (AURA) and Global Association of Indian Medical Students (GAIMS) supported the data collection. One of the authors contacted the student executives of GAIMS of various colleges and they in turn, disseminated it among their students. A video was made discussing the objectives of the study to give a clear idea to the volunteers.
Statistical analysis
Responses were received to Google account and collected as MS Excel sheet. Data cleaning was conducted to remove double entries, incomplete responses, and to exclude responses from ineligible participants. Statistical analysis was performed using the Statistical Package for the Social Sciences (SPSS) version 24.0 (SPSS, Inc., Chicago, IL, USA). Tests of significance used were Chi-square test to determine the role of sociodemographic characteristics on relevant questions. The statistical significance level was set at P < 0.05 (two-sided) with a confidence interval of 95%.
Results
A total of 7875 responses were obtained from 200 colleges. After data cleaning, 7416 final responses were analyzed. Among the 7416 students, 51.8% were from government medical colleges and 48.2% were from private medical colleges.
The mean age of the students in the study was 20.7 + 1.49 years. About 51% were in the age group of 21–24 years, 48.2% were in the age group of < 20 years, and 1.8% were above 25 years of age. The majority of the study participants were females (63.3%) and 36.7% were males. About 31.5% of students belonged to 1st MBBS, 25.5% from 2nd MBBS, and 40.4% from 3rd MBBS Part I and II. Interns accounted for 2.6% of the study participants. There were more girls than boys in all the groups.
Regarding the perception on whether they should start practicing MCQs, the majority of students responded in the affirmative. A higher proportion of students from senior classes responded that they should start preparing MCQs to appear for competitive examinations such as National Exit Test (NEXT), National Eligibility cum Entrance Test (NEET) and United States Medical Licentiate Examination (USMLE) as compared to students from MBBS first and second years. This difference was found to be statistically significant [Table 1a].
Table 1a.
Association of student’s responses on various questions with respect to their current academic year
| Questions | Current academic year | Total | ||||
|---|---|---|---|---|---|---|
|
| ||||||
| 1st MBBS (%) | 2nd MBBS (%) | 3rd MBBS part 1 (%) | 3rd MBBS part 2 (%) | Internship (%) | ||
| Q1 | ||||||
| No | 240 (10.3) | 172 (9.1) | 122 (7) | 103 (8.2) | 8 (4.1) | 645 (8.7) |
| Yes | 2095 (89.7) | 1717 (90.9) | 162 (93) | 1148 (91.8) | 185 (95.9) | 6771 (91.3) |
| P | 0.001 | |||||
| Q2 | ||||||
| 1st MBBS | 1407 (63) | 725 (38.4) | 676 (38.7) | 479 (38.3) | 63 (32.6) | 3413 (46) |
| 2nd MBBS | 675 (28.9) | 809 (42.8) | 495 (28.3) | 328 (26.2) | 38 (19.7) | 2345 (31.6) |
| 3rd MBBS part 1 | 160 (6.9) | 302 (16) | 460 (26.3) | 248 (19.8) | 40 (20.7) | 1210 (16.3) |
| 3rd MBBS part 2 | 14 (0.6) | 18 (1) | 58 (3.3) | 120 (9.6) | 12 (6.2) | 222 (3) |
| Internship | 16 (0.7) | 35 (1.9) | 59 (3.4) | 76 (6.1) | 40 (20.7) | 226 (3) |
| P | 0.001 | |||||
| Q3 | ||||||
| No | 263 (11.3) | 282 (14.9) | 312 (17.8) | 223 (17.8) | 34 (17.6) | 1114 (15) |
| Yes | 2072 (88.7) | 1607 (85.1) | 1436 (82.2) | 1028 (82.2) | 159 (82.4) | 6302 (85) |
| P | 0.001 | 7416 | ||||
| Q4 | ||||||
| 250-500 | 673 (28.8) | 503 (26.6) | 424 (24.3) | 324 (25.9) | 54 (28) | 1978 (26.7) |
| 500-1000 | 507 (21.7) | 453 (24) | 493 (28.2) | 303 (24.2) | 50 (25.9) | 1806 (24.4) |
| >1000 | 399 (17.1) | 377 (20) | 353 (20.2) | 241 (19.3) | 35 (18.1) | 1405 (18.9) |
| Don’t know | 756 (32.4) | 556 (29.4) | 478 (7.3) | 383 (30.6) | 54 (28) | 2227 (30) |
| P | 0.001 | |||||
To the question of the timing of initiation of MCQs, the majority of students (46%) expressed that it should start from 1st MBBS itself. Only 6% stated that the preparation may start in 3rd MBBS part II or internship. The difference in perception among the different MBBS years was found to be statistically significant [Table 1a] [Figure 1]
Figure 1.
From which year they should start preparing for multiple-choice questions?
The majority of the students (85%) felt that MCQ-oriented preparation will be helpful for MBBS professional (theory and practical) examinations. It was observed to be around 88% among 1st MBBS and declined slightly in higher class students with 85% among 2nd MBBS students and 82% among 3rd MBBS students and interns. This difference was found to be statistically significant [Table 1a] [Figure 2].
Figure 2.
Distribution of study participants who felt multiple-choice questions-oriented preparation useful for MBBS Professional examinations
Regarding the number of MCQs which students should practice, after learning from the textbook, 30% responded that they were not sure, 24% opined that 500–1000 would be sufficient and 27% felt that 250–500 questions would be sufficient. Another 18% opined that 1000 MCQs need to be practiced. There were variations among the perceptions of students from the various MBBS years, and this difference among the different MBBS years was statistically significant [Table 1a] [Figure 3]
Figure 3.
Distribution of study participants based on number of multiple-choice questions to be prepared after reading the subject
To the question of whether MCQs are more concept-based or memory-based in USMLE as compared to NEET PG, 73.9% of students responded that they are concept based and another 24% responded that they do not know the difference. It was observed that 1st MBBS students (30.9%) responded that they do not know the difference as compared to the rest of the students. This difference was found to be statistically significant [Table 1b].
Table 1b.
Association of student’s responses on various questions with respect to their current academic year
| Questions | Current academic year | Total | ||||
|---|---|---|---|---|---|---|
|
| ||||||
| 1st MBBS (%) | 2nd MBBS (%) | 3rd MBBS part 1 (%) | 3rd MBBS part 2 (%) | Internship (%) | ||
| Q5 | ||||||
| Concept and case based | 1550 (66.4) | 1425 (75.4) | 1385 (79.2) | 966 (77.2) | 152 (78.8) | 5478 (73.9) |
| Don’t know | 722 (30.9) | 420 (22.2) | 330 (18.9) | 265 (21.2) | 40 (20.7) | 1777 (24) |
| Memory based | 63 (2.7) | 44 (2.3) | 33 (1.9) | 20 (1.6) | 1 (0.5) | 161 (2.2) |
| P | 0.001 | |||||
| Q6 | ||||||
| Concept and case based | 2135 (91.4) | 1699 (89.9) | 1594 (91.2) | 1135 (90.7) | 178 (92.2) | 6741 (90.9) |
| Memory based | 200 (8.6) | 190 (10.1) | 154 (8.8) | 116 (9.3) | 15 (7.8) | 675 (9.1) |
| P | 0.472 | |||||
| Q7 | ||||||
| 50 percent | 824 (35.3) | 535 (28.3) | 341 (19.5) | 280 (22.40 | 31 (16.1) | 2011 (27.1) |
| 60 percent | 573 (24.5) | 505 (26.7) | 446 (25.5) | 315 (25.2) | 53 (27.5) | 1892 (25.5) |
| 70 percent | 566 (24.2) | 499 (26.4) | 532 (30.4) | 404 (32.3) | 68 (35.2) | 2069 (27.9) |
| 80 percent | 372 (15.9) | 350 (18.5) | 429 (24.5) | 252 (20.1) | 41 (21.2) | 1444 (19.5) |
| P | 0.001 | |||||
| Q8 | ||||||
| No | 738 (31.6) | 652 (34.5) | 546 (31.2) | 515 (41.2) | 104 (53.9) | 2555 (34.5) |
| Yes | 1597 (68.4) | 1237 (65.5) | 1202 (68.8) | 736 (58.8) | 89 (46.1) | 4861 (65.5) |
| P | 0.001 | |||||
Q1. Should you also practice MCQ’s for competitive exams such as NEXT, NEET PG, USMLE? Q2. If yes for above question, from which year should a student should start solving MCQ? Q3. MCQ oriented preparation will be helpful for MBBS professional (theory and practical) examinations? Q4. How many numbers of MCQ’s should a student prepare after reading text book for each subject? (based on the current academic year) Q5. MCQ for NEET PG and USMLE are slightly different, what do you think USMLE focuses on? Q6. Which type of MCQs do you like? Q7. How many questions should be from clinical subjects in NEXT? Q8. MBBS professional theory examination are for 14 subjects in total. Do you prefer a change to 100% MCQ computer-based examination instead of the current essay and short answer, so that you are more prepared for NEXT and USMLE examination? MCQ’s=Multiple choice questions, NEXT=National exit test, USMLE=United States medical licentiate examination, NEET-PG=National eligibility cum entrance test postgraduate
Most of the students (90.9%) preferred that the MCQs should be concept and case scenario based. The preference for concept-based MCQs was 91.4%, 89.9%, 91.2%, 90.7%, and 92.2% among 1st MBBS, 2nd MBBS, 3rd MBBS part 1, 3rd MBBS part II, and interns, respectively. The differences among the students of various years were not statistically significant [Table 1b] [Figure 4].
Figure 4.
Multiple-choice questions preference based on case-based, concept-based, and memory-based
In the present study, 27.90%, 25.5% and 27.9% of the students opined that in the NEXT examination, MCQs from clinical participants should be 50%, 60% and 70% of total marks respectively. About 19.5% of students stated that clinical participants should be allotted 80% of marks. More 1st MBBS students (35.3%) preferred lesser allotment (50%) of MCQs from clinical participants as compared to the students from higher years. The differences between the opinions of the students of various MBBS years were found to be statistically significant [Table 1b] as 1st-year students will not have adequate exposure to the clinical participants [Figure 5].
Figure 5.
Perception of students regarding allotment of multiple-choice questions on clinical participants in NEXT exam
To the question about whether the current system of university theory examinations of essays and short notes be replaced in total with MCQs so that they are more prepared for NEXT and USMLE examinations, 65.5% responded with a “yes” and 34.5% said “no.” Lesser number of students in internship (46.1%) and 3rd MBBS part II (58.8%) opined that MCQs should replace the current method of theory examination as compared to 1st MBBS students (68.4%) and this difference was statistically significant [Table 1b].
Discussion
In the present study, the majority of students (85%) opined that MCQ-oriented preparation is useful for MBBS examinations. This is in contrast to a study among preclinical medical students by Bird et al.,[4] which revealed that 54% of students reported preferring a Context Rich Short Answer Questions (CR-SAQ) examination format over MCQs format.
The majority of the students in the present study expressed that MCQs should be case scenarios or concept based which tests higher cognitive learning. Moeen-uz-Zafar Khan and Badr Aljarallah[5] concluded in their study that a well-constructed MCQ is superior to Modified Essay Questions in testing higher cognitive skills.
With regard to the present examination pattern in the University Examinations, about 65.5% of students preferred a change to 100% MCQ computer-based examination instead of the current essay and short answer so that they are more prepared for NEXT and USMLE examination. Hift[6] debated that MCQ format is better than open-ended question format and suggested phasing out of open-ended question format in summative assessment.
The majority of the students are of the opinion that MCQ preparation should be from the first MBBS. A few studies have also been conducted where writing MCQs was given as a task to the students themselves and study its effect on the performance in exams. The studies reported that such activities had a positive influence on learning behavior in terms of developing deep learning (Grainger study[7] ).
Herroro et al.[8] concluded that MCQs may be a valuable tool for medical education as a learning tool. Bottemly et al.[9] observed in their study that involving medical students in preparing MCQs that the students are eager participants and produced a large repository of relevant, good quality MCQs. There is also a widely accepted fact that formulating good quality MCQs is a demanding assignment. Chandre et al.[10] in their study emphasized the need for faculty development programs on the creation of MCQs. Mitayani Purwoko et al.[11] in a study observed that 61.2% of MCQs were of recall method and 50% of questions are to be revised. They recommended a faculty development program. Scully et al.[12] in their paper reiterated that MCQs have the capacity to assess certain higher-order skills and also provided practical recommendations for test developers seeking to purposefully construct such items.
Limitation and recommendation
The study was a cross-sectional study done among medical students to assess perceptions about examination pattern only. Students were not assessed by means of MCQs nor was there any comparison between the students’ performance on different assessment tools.
The findings of this study may provide inputs to the regulatory authorities and other stakeholders instrumental in designing and implementing the medical curriculum. The use of MCQs in assessment may be used as a learning tool as well as an assessment method. This may be more widely used in undergraduate medical training.
Conclusion
The study participants included students from across the country. The majority of the students had a positive opinion about MCQs and that MCQs should be used from 1st year. Students preferred MCQs which assessed higher-order cognitive skills. They opined those clinical participants should get more weight-age in assessment.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
Acknowledgments
The authors would like to acknowledge, Dr. Abbas Ali, MS (Orthopedics), DNB, Assistant Professor, Department of Orthopedics, Osmania Medical College, Hyderabad; Dr. Jaswanth Tanakala, Postgraduate, Department of Medicine, NRI Medical College, Guntur; Sai Venkata Kalyani Ponnaganti, Rangaraya Medical College, Kakinada; Aradhya Abrol, Dr. Rajendra prasad government medical college, Tanda; Burugu Hemanth Reddy, Prathima Institute of Medical Sciences, Karimnagar, for their support and assistance in completing the research successfully.
References
- 1.National Medical Commission – Introduction. [Last accessed on 2020 Nov 25]. Available from: https://www.nmc.org.in/about-nmc/introduction .
- 2.Kumar R, Pal R. India achieves WHO recommended doctor population ratio: A call for paradigm shift in public health discourse! J Fam Med Prim Care. 2018;7:841–4. doi: 10.4103/jfmpc.jfmpc_218_18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. [Last accessed on 2020 Dec 04]. Available from: https://www.nmc.org.in/information%20desk/for-%20students-to-study-in-india/list-of-college-teachingmbbs .
- 4.Bird JB, Olvet DM, Willey JM, Brenner J. Patients don’t come with multiple choice options: Essay-based assessment in UME. Med Educ Online. 2019;24:1649959. doi: 10.1080/10872981.2019.1649959. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Khan MU, Aljarallah BM. Evaluation of Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) as a tool for assessing the cognitive skills of undergraduate medical students. Int J Health Sci (Qassim) 2011;5:39–43. [PMC free article] [PubMed] [Google Scholar]
- 6.Hift RJ. Should essays and other “open-ended”-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ. 2014;14:249. doi: 10.1186/s12909-014-0249-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Grainger R, Dai W, Osborne E, Kenwright D. Medical students create multiple-choice questions for learning in pathology education: A pilot study. BMC Med Educ. 2018;18:201. doi: 10.1186/s12909-018-1312-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Herrero JI, Lucena F, Quiroga J. Randomized study showing the benefit of medical study writing multiple choice questions on their learning. BMC Med Educ. 2019;19:42. doi: 10.1186/s12909-019-1469-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Bottomley S, Denny P. A participatory learning approach to biochemistry using student authored and evaluated multiple-choice questions. Biochem Mol Biol Educ. 2011;39:352–61. doi: 10.1002/bmb.20526. [DOI] [PubMed] [Google Scholar]
- 10.Chandra S, Katyal R, Chandra S, Singh K, Singh A, Joshi HS. Creating valid multiple-choice questions (MCQs) bank with faculty development of pharmacology. Indian J Physiol Pharmacol. 2018;62:359–36. [Google Scholar]
- 11.Mitayani P, Trisnawati M. Evaluating the use of MCQ as an assessment method in a medical school for assessing medical students in the competence-based curriculum. Indonesian J Med Educ. 2018;7:54. [Google Scholar]
- 12.Darina S. Constructing multiple-choice items to measure higher-order thinking. Pract Assess Res Eval. 2017;22:04. doi: 10.7275/ca7y-mm27. [Google Scholar]