Psychometric Evaluation of Forensic Medicine Multiple-Choice Questions: An Item Analysis From a Nepali Medical College

Abstract

Background

Multiple-choice questions (MCQs) are test items that are essential to assess the competency of medical students in forensic medicine. However, the psychometric quality of these test items in Nepali medical colleges remains largely unexplored. This study evaluated the difficulty index, discrimination index, and distractor effectiveness of forensic medicine MCQs to identify areas requiring improvement.

Methods

This cross-sectional study evaluated 30 MCQs from the end-of-semester forensic medicine examination which was administered to 100 third-year medical students. The item difficulty, the discrimination index (via Kelley’s 27% method), and performance of the distractor were calculated for each question.

Results

The students showed high performance, with a mean score of 23.25±3.61 out of 30. Difficulty analysis showed that majority of the items (60%) fell within an acceptable difficulty range (70-90% correct), while 16.7% were very easy and 6.6% were difficult. Discrimination was good in 43.3% of the items, moderate in 23.3%, and poor in 33.3%. Distractor analysis identified non-functional distractors in a large proportion of items, including nine items (30%) with at least one zero-response option. Overall, 44 of 90 distractors (48.9%) across all items were non-functional. A statistically significant negative correlation was found between item difficulty and discrimination index (ρ = −0.4265, p = 0.019); items of moderate difficulty (50-69% correct) showed the highest mean discrimination, with both moderate- and acceptable-difficulty items outperforming items at the extremes of difficulty.

Conclusions

The examination revealed adequate student performance; however, item quality analysis demonstrated the need for improvement, with approximately one-third of the items failing to meet the desired standard and therefore requiring revision due to poor discrimination or non-functional distractors. Structured faculty training in item-writing principles and regular psychometric evaluation may help improve assessment quality in Nepali medical education.

Introduction

Multiple-choice questions are widely used in medical education for their objectivity, scalability, and ability to assess a wide range of knowledge in the cognitive domain.^1,2 However, the validity and reliability of MCQ-based assessments are critically dependent on the psychometric quality of individual items. Item analysis, a systematic evaluation of question difficulty, discrimination power, and distractor effectiveness, identifies flawed items and informs evidence-based revisions.^3-6 Poorly constructed MCQs with non-functional distractors and ambiguous stems can artificially inflate scores through guessing, fail to differentiate high- from low-performing students, and ultimately compromise assessment validity.^6,7 Given that forensic medicine is a core third-year subject at Lumbini Medical College (affiliated with Kathmandu University) with significant medico-legal implications, reliable assessment of student mastery in this discipline is essential.

Although MCQs are integral to end-of-semester exams, no systematic analysis of their psychometric properties has been conducted in forensic medicine in Nepal, which could compromise the quality of the assessment. Research that evaluates the quality of MCQ assessment in Nepali medical education is limited. Furthermore, no published studies have been conducted to assess the quality of assessment, specifically in forensic medicine, at Lumbini Medical College or any other institution in Nepal. A recent analysis of undergraduate forensic medicine curricula in Nepal revealed substantial inter-institutional variation in teaching methodologies and practical assessments, thereby highlighting the need for standardized quality assurance measures in both teaching and assessment. ⁸

Item writing often relies on the wisdom and experience of the assessment specialist ¹ ; therefore, systematic feedback based on psychometric data is essential for faculty development and curricular improvement. Identifying and revising flawed items improves assessment fairness, alignment with learning objectives, and provides actionable data for faculty training. ² High-quality MCQ assessment supports improved student evaluation, more accurate determination of learning outcomes, and curricula that are better aligned with institutional and national standards.^2,9

The present study aims to bridge this gap in the assessment of Nepali medical education by systematically analyzing the MCQs of the forensic medicine end-of-semester examination at Lumbini Medical College. The specific objectives are as follows:

• i. To determine the item difficulty index of each MCQ to identify questions that are well-calibrated, overly easy, or disproportionately difficult;

• ii. To calculate the discrimination index of each item to assess whether the questions effectively differentiate between high- and low-performing students;

• iii. To evaluate distractor effectiveness by identifying non-functional distractors and assessing whether each distractor attracts appropriate proportions of student responses;

• iv. To identify items that necessitate revision on the basis of psychometric analysis and to formulate evidence-based recommendations to improve assessment quality and fairness;

• v. To provide evidence-based guidance to inform faculty development programs and provide institution-specific insights that may inform future multi-institutional research.

Methods

This cross-sectional study was conducted between September and December 2025. Lumbini Medical College in Palpa, Nepal, is affiliated with Kathmandu University. Forensic medicine is taught during the third year of medical school (the sixth and seventh semesters). At the end of each semester, students must take the summative evaluation administered by their medical college in order to qualify for the university examination. Forensic medicine is a separate entity taught during the third year, with a weighted mark of one hundred for the theoretical component and one hundred for the practical component. The one hundred marks for theory are distributed as thirty marks for objective-type questions, fifty marks for subjective-type questions, and twenty marks carried from internal assessments. For the objective type, there are 30 multiple-choice questions (MCQs), each worth one mark. Each question consists of a stem with four options (A, B, C, and D), of which one is the correct answer and the remaining three are distractors. The MCQs are designed as single correct answer type questions, where one option is definitively correct, and the distractors are incorrect responses. Each question has exactly one correct answer; there is no negative marking. No formal test blueprint was used to map items to predefined learning objectives, content domains, or cognitive levels (e.g., Bloom’s taxonomy), which may influence the interpretation of item performance. During the examination, students marked the correct answer on the ‘optical mark recognition’ (OMR) sheet. At the end of the first 30 minutes of the three-hour examination, the students submit both the OMR sheet and the question paper and then proceed to answer the subjective-type questions.

All 100 third-year medical students enrolled at Lumbini Medical College who appeared for the forensic medicine end-of-semester examination in 2025 were included. Students who were absent from the examination were excluded, although no absences were recorded in this cohort. The OMR sheet of each student was collected and the response to each question as marked by the student in the OMR was entered into the Excel sheet. To ensure data precision, all OMR responses were entered by one investigator (AA) and independently verified by a second investigator (SN) against the original answer sheets. Any discrepancies were resolved by returning to the physical OMR sheet. After verification, student roll numbers were removed from the spreadsheet to ensure anonymization before analysis commenced. Three items (Q8, Q25, and Q26) each had one student who did not mark any response, yielding 99 recorded responses for these items. For the purpose of calculating item difficulty and discrimination, the missing response of each item was scored incorrect, so the denominator (N) was fixed at 100 for every item. This approach is consistent with the summative evaluation scoring policy, where unanswered questions receive no credit. Because the non-respondent did not select any option, the option-level responses reported for these three items sum to 99 rather than 100.

The questions numbered 1 to 30, each with four options marked as A, B, C, and D were analyzed. We analyzed the individual questions for correct answers, difficulty ratings, and distractors. The calculations were performed in an Excel spreadsheet, and the results are expressed as percentages. The item difficulty index (p) was calculated as the proportion of students who answered the item correctly:

$$\text{Item}\hspace{0.17em}\text{difficulty}\hspace{0.17em}\text{index}\hspace{0.17em}\left(\mathrm{p}\right)=\frac{\text{Number}\hspace{0.17em}\text{of}\hspace{0.17em}\text{students}\hspace{0.17em}\text{answering}\hspace{0.17em}\text{correctly}}{\text{Total}\hspace{0.17em}\text{number}\hspace{0.17em}\text{of}\hspace{0.17em}\text{students}\hspace{0.17em}\text{responding}}$$

Items were classified as: very easy (p > 0.90), acceptable difficulty (p = 0.70–0.90), moderate difficulty (p = 0.50–0.69), and difficult (p < 0.50).

The distractor effectiveness was evaluated by examining the proportion of students who selected each incorrect option. A distractor was considered functional if it attracted ≥5% of student responses and non-functional if it attracted <5% or zero responses.

$$\text{Distractor}\hspace{0.17em}\text{Proportion}=\frac{\text{Number}\hspace{0.17em}\text{of}\hspace{0.17em}\text{students}\hspace{0.17em}\text{selecting}\hspace{0.17em}\text{the}\hspace{0.17em}\text{option}}{\text{Total}\hspace{0.17em}\text{number}\hspace{0.17em}\text{of}\hspace{0.17em}\text{students}\hspace{0.17em}\text{responding}}\times 100$$

To calculate the discrimination index, students were ranked according to their total score on the exam. Following Kelley’s (1939) method, the top 27% (n=27) and bottom 27% (n=27) of students by total score were identified as the high-performing and low-performing groups, respectively. This approach maximizes statistical differentiation between groups and is a well-established method in item analysis. ¹⁰ The discrimination index (DI) for each item was then calculated as follows:

$$\text{Discrimination}\hspace{0.17em}\text{Index}\hspace{0.17em}\left(\text{DI}\right)=(P_{\mathit{high}}-P_{\mathit{low}})/N_{\mathit{group}}$$

where P_high and P_low are the number of correct responses in the high and low groups respectively, and N_group = 27. Internal consistency reliability was assessed using the Kuder-Richardson Formula 20 (KR-20), which is applicable to dichotomously scored MCQ examinations.

Statistical analyses were conducted in R (version 4.4.1, R Foundation for Statistical Computing, Vienna, Austria). Descriptive statistics (means, standard deviations, and percentages) were computed for all MCQs. Item difficulty, discrimination index, and distractor effectiveness were subsequently calculated for each item as described above. To examine the relationship between item difficulty and discrimination index, a Spearman rank correlation coefficient (ρ) was calculated. Spearman’s correlation was chosen over Pearson’s because with only 30 items, normality of both variables cannot be assumed, and Spearman’s is more robust to outliers and non-normal distributions in small samples. A two-tailed significance level of α = 0.05 was used for all statistical tests. Additionally, items were stratified by difficulty category (very easy, acceptable, moderate, and difficult), and mean discrimination indices were compared across categories. Because this record-based study used only anonymized answer sheets and involved no direct participant contact, the ethics committee waived the requirement for informed consent.

Results

All 100 OMR sheets were received, confirming that no students were absent from the examination. All the questions were answered except three, numbered 8, 25, and 26, which had 99 responses.

Student Performance Analysis

The mean score was 23.25±3.61 out of 30. The internal consistency of the examination was KR-20 = 0.67, indicating moderate reliability. While this value approaches the acceptable threshold of 0.70 for classroom assessments, it falls below the 0.80 benchmark recommended for high-stakes summative evaluations. This is consistent with the identified item flaws, specifically, the high proportion of non-functional distractors and the 10 items with poor discrimination which reduce inter-item consistency. Reliability may improve once underperforming items are systematically revised. This expectation is based on theoretical reasoning rather than a recalculated KR-20 on a revised item set, and should be confirmed empirically (e.g., by re-estimating KR-20 after piloting the revised items) in future works. The median score (50^th percentile) was 24, or 80% of the test score, which may reflect both student performance and item characteristics. Forty-one students (41%) scored 25 or higher, demonstrating excellent mastery of the subject. There were only two students who scored less than 50% (<15 marks), highlighting that most of the students achieved minimum competency.

Figure 1 shows the distribution of the marks with a left-skew (-0.964) with a peak performance cluster at approximately 24-27. The tight score distribution is reflected in a narrow interquartile range of only 5 points (Q1 = 21, Q3 = 26), indicating that most students performed within a narrow band. The fact that 3 students achieved near-perfect scores (29/30) indicates that the exam was appropriately challenging yet achievable.

Figure 1.

Distribution of student performance on forensic medicine MCQ examination (N=100). The left skewed distribution (skewness = −0.964) has a median of 24 marks (80%)

Item Analysis: Key Insights for Question Quality Assessment

Five items (16.7%) had >90% correct responses (Table 1). A high proportion of correct responses may reflect well-mastered, clearly worded items, but may equally indicate items that were too straightforward to differentiate among students. So, difficulty alone does not establish that an item is well constructed. The majority of the items (60%, n=18) fell within the acceptable difficulty range (70–90% correct). Acceptable-difficulty items are generally well positioned to discriminate between high- and low- performing students, but difficulty range alone does not guarantee item quality. Five items (16.7%) had 50–69% correct responses.

Table 1.

Item Analysis of the MCQs

Question	Option A	Option B	Option C	Option D	Correctly answered	Difficulty level
1	1	97	0	2	97	Very easy
2	62	2	2	34	62	Moderate
3	5	6	77	12	77	Acceptable
4	4	89	4	3	89	Acceptable
5	3	8	84	5	84	Acceptable
6	2	84	12	2	84	Acceptable
7	95	4	1	0	95	Very easy
8	6	90	0	3	90	Acceptable
9	95	2	0	3	95	Very easy
10	79	6	15	0	79	Acceptable
11	83	6	4	7	83	Acceptable
12	9	6	8	77	77	Acceptable
13	4	26	6	64	64	Moderate
14	4	1	1	94	94	Very easy
15	1	3	81	15	81	Acceptable
16	6	85	7	2	85	Acceptable
17	10	3	81	6	81	Acceptable
18	10	14	10	66	66	Moderate
19	2	4	70	24	70	Acceptable
20	10	3	85	2	85	Acceptable
21	0	0	96	4	96	Very easy
22	74	7	18	1	74	Acceptable
23	12	13	24	51	24	Difficult
24	7	3	21	69	69	Moderate
25	8	8	83	0	83	Acceptable
26	37	14	44	4	37	Difficult
27	5	84	10	1	84	Acceptable
28	12	2	27	59	59	Moderate
29	80	7	13	0	80	Acceptable
30	13	82	0	5	82	Acceptable

MCQs: Multiple-choice questions. Correctly answered = number of students (out of 100) selecting the correct option; the denominator was fixed at 100 for all items, with any unanswered item scored as incorrect. Difficulty level categories: Very easy >90%, Acceptable 70–90%, Moderate 50–69%, Difficult <50%. For Q8, Q25, and Q26, one student did not select any option; because that non-response is not attributable to any of the options A-D, the four option counts shown for these items sum to 99 rather than 100.

Initially, Q27 had only one single response (option D) correctly marked per the answer key provided, which was a contrasting finding. We then returned to the question and identified that the correct option was option B, and the answer key had a typographical error: ‘B’ was mistyped as ‘D’. The corrected key (option B) was used throughout the present psychometric analysis. The correction was also applied to students' official scores before the results were published.

The easiest question was Q1 (97% correct), and the most difficult question was Q23 (24% correct), with a mean difficulty of 76.1%. The percentage of questions in the acceptable range was 60% (18/30 questions).

Question 23 had only 24% correct answers, with students choosing option D (51%) instead of the correct option C. This pattern is consistent with a problematic item, but the qualitative data alone cannot establish whether the underlying cause is the wording or structure of the stem, genuine ambiguity was observed between options C and D, or insufficient instructional coverage of the underlying concept; distinguishing between these possibilities would require expert content review or qualitative follow-up (e.g., faculty review of the item or student feedback).

Question 26 had only 37% correct responses, with answers split between option A (37%, correct) and option C (44%). This near-even split is consistent with ambiguous wording or two arguably defensible options. But, as with Q23, the response-distribution data alone cannot confirm the specific cause.

The results indicate that:

• • Question 23 needs immediate revision.

• • Question 26 needs clarification.

• • Items with <50% or >90% success rates should be considered for omission or replacement.

• • Items combining acceptable-to-moderate difficulty, good or moderate discrimination, and functional distractors are strong candidates for retention; of the 18 acceptable-difficulty items, only 2 (Q2 and Q12) met this combined standard. Three items, (Q20, Q29, and Q30) showed poor discrimination despite acceptable difficulty, and the remaining 13 items had adequate difficulty and discrimination but at least one non-functional distractor.

The item difficulty analysis indicated that 5 items (16.7%) were very easy (>90% correct responses), 18 items (60%) fell within the acceptable difficulty range (70–90%), 5 items (16.7%) were of moderate difficulty (50–69%), and 2 items (6.6%) were difficult (<50%). A detailed distribution is provided in Table 2.

Table 2.

Difficulty Level of MCQs

Difficulty level	Criterion (% correct)	No. of items	% Of items	Item numbers
Very easy	> 90%	5	16.7%	Q1, Q7, Q9, Q14, Q21
Acceptable difficulty	70–90%	18	60%	Q3, Q4, Q5, Q6, Q8, Q10, Q11, Q12, Q15, Q16, Q17, Q19, Q20, Q22, Q25, Q27, Q29, Q30
Moderate difficulty	50–69%	5	16.7%	Q2, Q13, Q18, Q24, Q28
Difficult	< 50%	2	6.6%	Q23, Q26

% correct: percentage of students answering the item correctly.

Analysis of Non-functional Distractors

Distractor analysis revealed a critical issue in MCQ construction. There were nine questions (Questions 1, 7, 8, 9, 10, 21, 25, 29, and 30) that had 0 responses for an option. This signals that one of the three distractors to these questions was completely non-functional, and the students immediately recognized them as implausible.

Furthermore, Questions 2, 4, 5, 6, 11, 13, 14, 15, 16, 17, 19, 20, 22, 24, 26, 27, and 28 were found to have ineffective distractors, with several options across these items attracting fewer than five student responses. For example:

• • Question 2: Correct answer Option A (62), Strong distractor: Option D (34), Ineffective distractors: Options B (2), and C (2).

• • Question 19: Correct answer Option C (70), Strong distractor: Option D (24), Ineffective distractors: Options A (2), and B (4).

• • Question 26: Correct answer Option A (37), Strong distractor: Option B (14), and C (44), Ineffective distractor: Option D (4).

• • Question 28: Correct answer Option D (59), Strong distractors: Options A (12) and C (27), Ineffective distractor: Option B (2).

Across all 30 items, 44 of the 90 distractors (48.9%) met the <5% criterion for non-functional distractors and were not confined to the nine items with a fully unused (zero-response) option. This confirmed that distractor weakness was pervasive. These questions reduced difficulty because students could eliminate obviously implausible options, making it easier to answer correctly. Such items produced poor discrimination because they effectively functioned as two- or three-option MCQs rather than the intended four-option format, thereby increasing the probability of correct guessing. The use of such questions in an examination does not accurately measure student knowledge.

This analysis highlights the following:

• • Replace non-functional distractors.

• • Revise implausible options that attracted <5% of the responses.

• • As a general heuristic, well-functioning distractors are sometimes considered to ideally draw roughly 10–15% of responses each under a four-option format. The more commonly applied and operational criterion in item-analysis practice, and the one used for classification in this study, is the ≥5% functional threshold.

• • Review distractor quality – Distractors should be plausible to students who do not know the material well.

The discrimination index (DI) was calculated for all thirty items via Kelley’s 27% method (Table 3). Among the items analyzed, 13 (43.3%) had good discrimination (DI > 0.30), 7 (23.3%) had moderate discrimination (0.20 ≤ DI ≤ 0.30), and 10 (33.3%) had poor discrimination (DI < 0.20). These findings suggest that nearly half of the items could effectively differentiate between high- and low-performing students, whereas one-third of the items require revision or replacement to strengthen the overall performance of the assessment.

Table 3.

Discrimination Index of the MCQs Calculated Using Kelley’s 27% Method (Top and Bottom 27% of Performers, N=27 Each)

Question	Discrimination index	Category
Q1	0.074	Poor discrimination
Q2	0.407	Good discrimination
Q3	0.370	Good discrimination
Q4	0.222	Moderate discrimination
Q5	0.407	Good discrimination
Q6	0.370	Good discrimination
Q7	0.074	Poor discrimination
Q8	0.259	Moderate discrimination
Q9	0.148	Poor discrimination
Q10	0.333	Good discrimination
Q11	0.333	Good discrimination
Q12	0.593	Good discrimination
Q13	0.222	Moderate discrimination
Q14	0.185	Poor discrimination
Q15	0.222	Moderate discrimination
Q16	0.259	Moderate discrimination
Q17	0.519	Good discrimination
Q18	0.407	Good discrimination
Q19	0.444	Good discrimination
Q20	0.185	Poor discrimination
Q21	0.074	Poor discrimination
Q22	0.296	Moderate discrimination
Q23	0.037	Poor discrimination
Q24	0.593	Good discrimination
Q25	0.407	Good discrimination
Q26	0.370	Good discrimination
Q27	0.222	Moderate discrimination
Q28	0.185	Poor discrimination
Q29	0.185	Poor discrimination
Q30	0.185	Poor discrimination

DI: Discrimination Index, calculated using Kelley’s 27% method. Categories: Good DI >0.30; Moderate DI 0.20–0.30; Poor DI <0.20.

Table 4 stratifies mean discrimination indices by difficulty category. Items in the moderate difficulty range (50–69%) showed the highest mean DI (0.363), followed closely by acceptable-difficulty items (70–90%, mean DI = 0.323), then difficult items (<50%, mean DI = 0.203), and finally very easy items (>90%, mean DI = 0.111), which had the lowest mean discrimination of any category. Both moderate- and acceptable-difficulty items substantially outperformed the extreme categories (very easy and difficult) in capacity to discriminate. Correlation analysis between item difficulty and discrimination capacity showed a statistically significant negative correlation (ρ = −0.4265, p = 0.019) (Table 5).

Table 4.

Mean Discrimination Index by Difficulty Category

Difficulty category	n	Mean DI	SD	Interpretation
Very easy (>90%)	5	0.111	0.052	Poorest discrimination
Acceptable (70–90%)	18	0.323	0.119	Good discrimination
Moderate (50–69%)	5	0.363	0.165	Highest mean discrimination
Difficult (<50%)	2	0.203	0.235	Low/variable discrimination

DI: Discrimination Index. SD: Standard deviation. n: number of items in each difficulty category.

Table 5.

Spearman Rank Correlation Between Item Difficulty and Discrimination Index

Statistic	Value
Number of items (n)	30
Spearman’s rank correlation (ρ)	−0.4265
p-value (two-tailed)	0.019*
95% Confidence Interval	−0.718 to −0.066
Statistical significance	Significant (p < 0.05)
Effect size interpretation	Moderate negative correlation

*p < 0.05.

ρ: Spearman’s rank correlation coefficient. CI: Confidence Interval. p < 0.05.

Discussion

The strong learner performance observed in this study is consistent with data from Thailand, where fifth-year clinical students correctly answered 76.09% of MCQs in forensic medicine. ¹¹ This performance was markedly higher than the 62.94% recorded among third-year preclinical students. ¹¹ These outcomes support integrating forensic medicine into the clinical curriculum when students already possess sufficient basic medical knowledge and can rapidly apply clinical reasoning to medico-legal scenarios. ¹¹

Our findings also closely mirror an earlier item analysis conducted of the All India AYUSH Post Graduate Entrance Examination (AIAPGET) 2019. ¹² That study analyzed 100 MCQs from 14,229 candidates and found a mean difficulty index of 37.32±16.11% and a discrimination index of 0.46±0.27, with 85.4% functional distractors. ¹² Although our assessment focused on the summative evaluation of learned material, AIAPGET’s lower difficulty index was appropriate to differentiate a large pool of candidates competing for limited post-graduate seats. ¹² By contrast, our end-of-semester examination appears to assess core concepts of forensic medicine, though interpretation is limited by item flaws. The AIAPGET analysis also revealed 14.6% non-functional distractors. ¹² Together, these findings point to a common challenge in writing plausible distractors across South Asian contexts and support systematic faculty training in item-writing. ¹³

Studies in the Gulf region show that 27.5-49.7% of MCQs contain at least one flaw, generally the use of negative stems (“except”/”not”), unfocused stems, and non-functional distractors that suggest these are universal challenges in MCQ construction. These studies support that structured faculty training and rigorous peer review could help address these challenges.^14-16

A Saudi Arabian study similarly reported a distractor efficiency of 81.25% which indicated that most of the distractors were functional. ¹⁷ A study from the Arabian Gulf region by Kheyami et al., concluded that continuous training in item-writing will improve distractor efficiency. ¹⁸ Their study reported that 48.4% of the MCQ items had zero non-functional distractors and demonstrated a gradual improvement in distractor efficiency from 70.88% to 86.88% over four years through continuous quality improvement activities by their examination committee. ¹⁸ In Nepal, once a post-graduate student passes the final examination in forensic medicine, they join as faculty members at a medical college. Without any formal training in pedagogy or assessment design, they automatically qualify to become teachers as well as item-writers once they complete their postgraduate degree. As a consequence, the MCQs they develop do not meet the quality benchmark. Poorly constructed distractors and items that test only factual recall instead of higher-order thinking remain a persistent problem worldwide as they limit the validity and reliability of assessments. ³ Haladyna et al., reviewed 27 education assessment textbooks and 27 studies from the 1990s onward and put forth 31 validated guidelines to help teachers and test writers to construct better test items. ¹ A single training session for newly appointed medical faculty saw a significant improvement in MCQ quality in India (p=0.004) as reported by Vinay et al.,. ¹⁹ This improvement was most obvious in technical aspects such as clear directions, correct marking of negative stems, and reduction of grammatical errors. ¹⁹ The success of longitudinal faculty development programs has been demonstrated in multiple contexts. Studies from Pakistan and Saudi Arabia show how a structured program with repeated practice, peer review and feedback significantly improved the quality of MCQ and had a sustained impact on faculty knowledge and behavior.^5,20 Persistent challenges identified in this study can be addressed by a structured and sequentially designed faculty development programs that build item-writing skills in stages. ²⁰

Tarrant and Ware ⁶ reported that flawed MCQs tend to penalize high-achieving students while benefiting borderline students, a pattern that may explain the elevated median scores and narrow interquartile range observed in our examination. Targeted item review is therefore important for ensuring assessment fairness across all performance levels. This supports the case for directing early-career faculty development toward training in evidence-based item-writing.

On closer examination, the subpar DI observed in one-third of the items is not random but reflect an underlying, internally consistent psychometric pattern. Items that had accuracy rates of more than 90%, by definition, are poor discriminators. It is because if nearly the entire cohort answers correctly, the resulting ceiling effect mathematically drives the DI toward zero. Five such items in our assessment (Q1, Q7, Q9, Q14, and Q21) met this threshold, most likely because they tested concepts that had been extensively rehearsed or were otherwise insufficiently demanding. If such items are left unrevised; they artificially raise the overall pass rate without contributing anything to the test’s diagnostic precision. The high prevalence of non-functional distractors also attenuated discrimination that functionally reduced four-option MCQs to two- or three-option items; and also diminish the cognitive demand required to eliminate incorrect choices. This distractor failure, rather than flawed item stems per se, was the main mechanical reason for the weak discrimination observed in our dataset. Both mechanisms, very-easy items and non-functional distractors, reduced item variance and item-total correlation, which suppressed rather than inflated the overall KR-20 coefficient. This is consistent with our observed coefficient of 0.67, which fell short of the 0.80 benchmark recommended for high-stakes examinations. The observed patterns suggest that the high aggregate scores may partly reflect limitations in item construction rather than students’ mastery of the material alone. However, since this study did not include an independent criterion measure of student learning (e.g., an external standardized test or an expert-validated reference assessment), this remains a plausible explanation rather than a confirmed finding. It cannot exclude the possibility that the underlying mastery of the students was genuinely high.

The moderate negative correlation observed between item difficulty and discrimination index (ρ = −0.4265, p = 0.019) aligns with the classical test theory, which predicts reduced discriminatory power at the extremes of the difficulty continuum. Very easy questions are answered almost correctly by everyone, so distinguishing good performers from poor ones is not possible. Likewise, very difficult questions are answered by none, making discrimination, again, difficult. On average, easier items discriminated less effectively than harder ones. Within moderate-to-acceptable range, moderate-difficulty items (50–69%) had marginally higher mean discrimination than acceptable-difficulty items (70–90%). The gap between moderate- and acceptable-difficulty items shows that this relationship is not strictly linear across all difficulty tiers. The items at the extremes, very easy and difficult, showed the lowest discrimination, consistent with classical test theory predictions.

Careful review of defective MCQs, rather than their outright removal, helps preserve content coverage. ²¹ This approach guided the revision of problematic items, such as Q23 and Q26, and reflects a broader post-analysis revision process that helps maintain quality within the item bank.

AI-assisted item generation is increasingly used to support item development; however, human review remains essential.^22-24 A recent comparison of multiple-choice items generated by different AI-based chatbots for medical imaging found measurable differences between platforms in item applicability, difficulty, and discrimination indices. ²⁵ This underlines the need for expert content review and psychometric validation before AI-generated items are deployed in summative evaluations. Given that this study did not evaluate AI-generated items, further discussion is beyond the scope of this study.

Holland et al. (2015) reported that incorporating images into MCQ stems in histology did not significantly alter item difficulty or discrimination, a finding that supports the use of images to improve authenticity without compromising psychometric quality. ²⁶ Visual elements could be incorporated into forensic medicine MCQs to better test medico-legal interpretation skills if they adhere to evidence-based item-writing principles. Objective structured practical examinations (OSPE) in forensic medicine are a well-accepted complementary assessment method in Nepal to assess practical skills that MCQs cannot evaluate on their own. ²⁷

Our analysis was limited to MCQs from a single end-of-semester examination administered to a cohort of third-year medical students at a single institution. This constraint limits the generalizability of the findings in academic years, different cohorts of students, or other Nepali medical colleges. The difficulty of the item, discrimination, and distractor performance may vary between test cycles and institutional contexts. In addition, the homogeneous learner population may not reflect the performance patterns found in other settings of medical education. Furthermore, this study did not assess pedagogical alignment (whether items were explicitly mapped to learning objectives), structural integrity (adherence to established item-writing guidelines), or cognitive complexity (the level of thinking demanded by each item, e.g., recall versus application per Bloom’s taxonomy). These omissions limit the comprehensiveness of the psychometric evaluation. Future studies should incorporate blueprint mapping of learning outcomes and cognitive level classification alongside statistical item analysis. The KR-20 reliability coefficient of 0.67, although approaching acceptable levels for classroom assessments, did not meet the ≥0.80 threshold recommended for high-stakes summative exams. This limitation is partly attributable to the high proportion of items with poor discrimination and non-functional distractors identified in this study and will likely improve through focused item revision. To mitigate these limitations, future efforts should incorporate repeated exam cycles and multi-institutional comparisons to substantiate these observations and determine if the identified item-writing flaws are consistent patterns or exam-specific discrepancies.

Conclusion

The item analysis of forensic medicine end-of-semester MCQs at Lumbini Medical College revealed that while students demonstrated a high overall pass rate, the psychometric quality of the examination was mixed. The left-skewed score distribution and narrow interquartile range, observed alongside a KR-20 of 0.67 and a high prevalence of non-functional distractors, raise the possibility that strong aggregate scores may partly reflect item-construction weaknesses rather than genuine student mastery alone. However, in the absence of an independent criterion measure of learning, this study cannot directly confirm or rule out students’ true level of mastery, and this interpretation should be treated as a plausible hypothesis rather than an established conclusion. Poorly constructed items likely compressed the score range: by allowing weaker students to answer non-functional-distractor items correctly through elimination, while offering little additional room for stronger students to distinguish themselves, such items may have artificially narrowed the gap between high- and low-performing students.

Approximately one-third of the items require revision or replacement because of poor discrimination, non-functional distractors, or both. The statistically significant negative correlation between item difficulty and discrimination confirms that easier items tended to discriminate less effectively overall which is consistent with classical test theory. Within this dataset, items of moderate difficulty (50-69%) showed the highest mean discrimination, with acceptable difficulty items (70–90%) showing comparably strong discrimination. Both substantially outperformed items at the extremes of difficulty (very easy and difficult). This pattern should guide future item refinement, alongside distractor performance, in retention decisions.

Distractor analysis points to a need for faculty training in evidence-based item writing principles, as non-functional distractors and poorly plausible options remain prevalent. Retention of well-performing items and revision or removal of underperforming ones will enhance the reliability and validity of the test. Future examination cycles should include systematic piloting of new items, faculty training in item construction, collection and reflection on qualitative feedback, and expanded cohort sampling to improve generalizability.

Our findings deliver institution-specific, validated parameters for MCQ quality at one Nepali medical college. Whether these findings reflect broader patterns across Nepal’s medical colleges awaits verification through broader multi-institutional collaborative studies. This study generates institution-specific psychometric benchmarks and a quality-improvement framework that could inform faculty development efforts elsewhere. This study should be interpreted as a local quality improvement analysis rather than evidence of generalizable patterns. Broader conclusions require multi-institutional and longitudinal data.

ORCID iDs

Alok Atreya https://orcid.org/0000-0001-6657-7871

Devi Ram Acharya https://orcid.org/0000-0002-7295-6699

Samata Nepal https://orcid.org/0000-0001-9189-4510

Laxmi Prasad Sapkota https://orcid.org/0000-0001-9811-3879

Ritesh G. Menezes https://orcid.org/0000-0002-2135-4161

Bal Chandra Luitel https://orcid.org/0000-0002-5483-2385

Ethical Considerations

This study was approved by the Institutional Review Committee of Lumbini Medical College (IRC-LMC) vide letter reference number 14/25.00. This study adhered to the ethical principles outlined in the Declaration of Helsinki, ensuring the protection of participant data through anonymization and institutional ethical oversight.

Consent to Participate

Informed consent to participate was waived by the Institutional Review Committee of Lumbini Medical College (IRC-LMC) due to the retrospective, record-based nature of the study, which involved anonymized data and posed no risk to participants. This waiver is in accordance with national regulations and the committee’s approval (reference number 14/25.00).

Data Availability Statement

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.*