Table 3.

Representative comments on the drawbacks of having an essay-based exam form from students and faculty.

Theme	Student Comments	Faculty Comments
Acceptability	‘While I do not think there are “drawbacks” to essay based format, I think that they don’t help as much with the format of the step 1 exam. However I think that essay exams overall enhance learning, and learning should not be tailored to the format of the step one exam.’	‘This format is different from the USMLE exams that students will need to take, so it would be important to also make sure students feel adequately prepared for national standardized exams that are MCQ based.’
	‘Essay based exams take a much longer time to do, and can only assess a small amount of content that may not be reflective of the course.’	‘ … a multiple choice exam would be able to cover many more topics with more questions, because each question would take less time to answer and score. Also, it would better prepare the students for the type of exam they take for NBME.’
	‘The main drawback of an essay-based exam format is that it creates much higher stress for the students when studying.’	‘It is difficult to cover the range of information covered in the course.’
		‘I imagine this makes for a grueling testing experience for students.’
Construct Validity	‘Questions are often not completely precise and clear (in terms of what is being asked and what level of detail is needed in the answer), so that without the benefit of a selection of answer choices, this means that a lot of time is often spent just trying to figure out what the question is asking and what will constitute a satisfactory response.’	‘Students can come with answers that may be correct but not on the rubric.’
Reproducibility	‘Grading is a lot more subjective than with multiple choice which has one correct answer.’	‘Although we score based on a rubric, because the students are forming their own answers, it allows for a great deal of subjectivity in the scoring.’ ‘There can be a much wider range in depth and detail of responses, leading to increased challenges in creating a rubric to objectively assess responses.’
Feasibility	‘There is not enough time to give fully thought out answers that are actually explanatory for one’s thought process.’	‘From the faculty perspective, the grading can be onerous, not just from the sheer amount of time it requires, but essay-based exams also require more thought being put into the questions themselves and into the grading/rubric writing process. Totally worth it, though.’
Cueing Effect	‘The drawbacks to essay-based exams is that it makes the student have to own the material instead of being able to recognize topics based on multiple choice answers.’