ABSTRACT
Written knowledge assessments for physicians in training typically involve multiple-choice questions that use a clinical scenario in a single-best-answer format. The Royal College of Physicians Part 1 MRCP(UK) examination includes basic sciences themes that are challenging to assess through a clinical scenario. A realistic clinical setting based on everyday clinical practice and integral to the question is the clearest demonstration that the knowledge being assessed is clinically relevant. However, without special attention to detail, the scenario in a clinical science question can appear redundant or artificial. Reading unnecessary material frustrates candidates and threatens the reputation of the assessment. In this paper we discuss why a clinical scenario is important for basic science questions and offer advice on setting realistic and plausible clinical scenarios for such questions.
KEYWORDS: Science, clinical scenario, single-best-answer, Part 1 MRCP(UK)
Introduction
The practical relevance of assessing a physician’s knowledge through a scenario-based single-best-answer approach1 is self-evident. Doctors make decisions on clinical scenarios (in relation to diagnosis, investigation and management) every day of their working lives, based on the balance of probabilities. However, a physician’s working knowledge of the science that underpins these clinical decisions may not be so obvious in day-to-day clinical practice and so the assessment of clinical science through the medium of written clinical scenarios is not necessarily so straightforward.
The 2015 MRCP(UK) Standards Advisory Review Group recommended reviewing the science-based content of the Part 1 examination to ensure its clinical relevance.2 Some would argue that a question requiring knowledge that cannot be set into a clinical scenario is, by definition, of little practical use and something a physician may not need to know. On the other hand, a working knowledge of certain aspects of clinical science – for example cell structure and of normal biochemistry – is so fundamentally important that it must remain essential knowledge for practising clinicians. Furthermore, it is not always possible to identify the principles of clinical science that may become important to today’s clinicians in the future.
Is a clinical scenario really necessary?
Simply adding a clinical scenario to a basic science question, though conveying face validity, does not automatically make the question clinically relevant. Unless the clinical scenario is essential to the question, it risks appearing artificial or contrived; candidates do not want to read unnecessary material. A clinical scenario should make explicit the clinical relevance of the science being assessed. It should also ensure that candidates seek the necessary information from within the scenario (reflecting real-life practice) and so apply their knowledge to a particular situation, rather than merely recalling a fact. Furthermore, increases in the complexity of a question ought to improve the statistical reliability of the question item, though surprisingly this is unproven.3
First-year medical undergraduates in Cardiff are assessed by questions requiring that they seek answers to science questions from within a scenario. This encourages them to apply their knowledge even at this stage to specific situations, something they will have to do repeatedly during every clinical encounter once they graduate.
Advice to question writers
Having noted the advantages of a clinical scenario for clinical science questions, what is the best advice for question writers? The questions should address topics that are common and important in routine clinical practice. Therefore a clinical scenario for a science question should ideally describe a situation where knowledge of a particular aspect of clinical science was key to a successful clinical outcome.
Science questions involving pathology and microbiology already fit well into clinical scenarios, since patients with infection or other disease will consult with doctors, and knowledge of these disciplines is clearly important for optimal patient management. However, fewer clinical encounters require the direct application of knowledge of, say, molecular or cell biology, despite the importance of a thorough grounding in these topics for understanding a patient’s illness. For example, how might we write a clinical scenario based around a candidate’s application of knowledge of a molecular receptor, a chromosomal centromere or the Golgi apparatus?
Simply adding a clinical scenario to an already self-contained science question gives an artificial result, with a scenario that is merely decorative and redundant (Box 1). Candidates become irritated if they have taken the trouble to read and understand several lines of a clinical scenario, but then find that they needed only the brief lead-in to answer the question. Each clinical scenario should be ‘essential’, be integral to the question and require candidates to apply their knowledge specifically to it (Box 1).
Box 1.
Cell biology questions.
| Question with a ‘redundant’ scenario. |
| A 32-year-old man with generalised weakness had a biopsy taken from his quadriceps muscle, which was examined using electron microscopy. The pathologist reported an abnormal appearance of the Golgi apparatus. |
| What is the main role of the Golgi apparatus? |
| A. controls amino acid breakdown B. controls lipid breakdown C. controls protein processing D. initiates protein assembly E. initiates protein breakdown |
| Answer C – controls protein processing |
| This is basic science question with a ‘redundant’ scenario; the candidate needs only to read the lead-in in order to answer the question. |
| Question with an ‘essential’ scenario |
| A 32-year-old man with generalised weakness underwent a muscle biopsy that identified an abnormality of the Golgi apparatus. |
| What cellular function was most likely to be abnormal? |
| A. amino acid breakdown B. lipid breakdown C. protein assembly D. protein breakdown E. protein processing |
| Answer E – protein processing |
| This is the same basic science question but now the scenario is ‘essential’; the candidate must read the whole question to identify the best answer. |
Box 2.
Biochemistry question.
| Question in multiple true/false format. |
| Lactase breaks down lactose into glucose and one other sugar. |
| What is the other sugar? |
| A. fructose B. galactose C. maltose D. mannose E. sucrose |
| Answer B – galactose |
| This is a traditional biochemistry question in multiple true/false format, requiring simply the recall of a fact. |
| Question in single-best-answer format. |
| An 18-year-old woman had lactose intolerance. A dietitian advised her to add lactase to fresh milk several hours before drinking it, explaining that lactase works by breaking down the lactose in milk into glucose and one other sugar. |
| What other sugar was most likely mentioned? |
| A. fructose B. galactose C. maltose D. mannose E. sucrose |
| Answer B – galactose |
| This question assesses the same knowledge but now through a clinical scenario. Candidates must read the whole scenario before applying their knowledge to that scenario, and answer in the ‘best of’ format that characterises a single-best-answer item. |
The cover tests
The traditional cover test
Experienced question writers use the ‘cover test’ instinctively: a good question should be answerable even when the answer options are concealed. A lead-in that asks, ‘Which of these statements is true?’ not only requires the candidate to see the options before answering – rather than anticipating the answer from the scenario – but also is likely to be a traditional true/false multiple choice question with a single correct answer, requiring mere factual recall.
Incidentally, starting the lead-in with ‘Which?’ implies that the choice must lie only within the five options presented (indeed the case in a written examination, though not in real life). A candidate cannot choose without seeing the available options; the question therefore fails the cover test. By contrast, starting the lead-in with ‘What?’ implies that the question is sufficiently well written for the candidate to be confident that the best option will be listed, without having to check first; it therefore passes the cover test.
The science question cover test
If the scenario is an integral part of the question, the question should not be answerable with the scenario covered (Box 1). For example, a question lead-in that asks, ‘What pancreatic secretion is predominantly responsible for fat digestion?’ can be answered without the scenario, requiring only factual recall. But a lead-in that asks, ‘Secretion of what pancreatic enzyme is most likely to be impaired?’ requires candidates to apply their knowledge to the clinical scenario, rather than merely to recall a fact.
Can every question have a clinical scenario?
While it is possible to invent a scenario for every science question, it is often difficult to make this an effective way to explore a candidate’s knowledge of that science. For example, in the case of the Krebs cycle or messenger RNA, the scenario could describe a patient with a condition relating to a genetic or cellular abnormality who asks the doctor about the body’s energy production or about the translation of genetic messages; however, simply answering a patient’s factual question reduces the task to one of factual recall, despite the presence of the scenario. It would be better to envisage a clinical situation that requires application of one’s knowledge of the Krebs cycle or messenger RNA to that specific situation.
Making science questions clinically relevant
Setting a science question in a clinical context can reassure candidates (and quality inspectors) of its relevance. For example, ‘What enzyme is most likely to be deficient in cystic fibrosis?’ can be simply changed to:
A 2-year-old boy was found to have cystic fibrosis. The parents asked the doctor how the illness had come about.
What enzyme deficiency was most likely mentioned?
Hints for writing science-based clinical scenarios
Anatomy and histology
Making anatomy questions clinically relevant is straightforward: the clinical assessment of an injury or the practical application of surgery clearly requires knowledge of normal anatomy.
A 67-year-old man underwent laparotomy for abdominal pain. He was found to have a gastric tumour eroding through the posterior wall of his stomach.
What adjacent organ was most likely affected?
A 19-year-old man reported numbness over the dorsolateral hand after being stabbed in the lateral aspect of the right upper arm.
What nerve was most likely affected?
Histology questions are also easy to build through a clinical scenario because any organ presented to a pathologist can also contain normal cells, included in the pathologist’s report.
A 42-year-old man was found to have an unexplained abnormality of his liver function tests. He underwent a liver biopsy that was reported as normal.
What cell type most likely predominated in the specimen? OR (with an image)
What structure does the arrow most likely indicate? OR
Which arrow best identifies a Kupffer cell?
Physiology
Questions about the normal menstrual cycle, pregnancy, embryology, growth or development can all be fitted into realistic clinical scenarios relatively easily, since patients commonly present with problems related to these. However, setting some physiology themes to a scenario is more challenging, despite the necessity of knowledge of normal physiology when assessing patients with disease. Scenarios to assess knowledge of normal physiology might describe a patient with a disease, focusing the question on the normal function that is lost.
A 25-year-old woman developed chronic renal failure from glomerulonephritis. The doctor inferred that damage to the kidney’s proximal convoluted tubule system had impaired a particular function of the kidney.
What normal function was most likely affected?
A 25-year-old woman developed chronic renal failure from glomerulonephritis. Her blood tests suggested that the glomerular filtration rate had fallen by 90% from normal.
What is the best estimate of her filtration rate?
Biochemistry, molecular and cell biology
These questions are the most difficult to write into a clinical scenario. A useful trick is to have a question based on a disease, in which the scenario discusses normal (or absent) biochemical or cell biology function (Boxes 1 and 2).
Clinical pharmacology
The prescribing of medications fits neatly into clinical scenarios, but devising a scenario to give clinical context to a question on a drug’s mechanisms of action, pharmacokinetics and pharmacodynamics requires more thought.
A 67-year-old man was taking a regular medication for congestive heart failure. The doctor identified this drug as a phosphodiesterase inhibitor, which prevents the inactivation of an intracellular second messenger in cardiac myocytes.
What is the most likely identity of this second messenger?
adenylyl cyclase
creatine kinase
cyclic AMP
cyclo-oxygenase
phospholipase
Answer C – cyclic AMP
Other options for scenarios relating to basic pharmacology may include a patient who has questions after reading the medication data sheet or after searching the internet, or who has concerns about a new drug interacting with an existing one.
Microbiology
Microbiology is relatively easy to assess through clinical scenarios, as these can feature patients with infectious diseases.
A 17-year-old girl underwent a brain biopsy to investigate a progressive dementia thought to be a late complication of infantile measles. The brain biopsy identified a virus showing characteristics of the measles virus.
What characteristic did the virus most likely have?
A 32-year-old man developed a Plasmodium malariae infection after visiting the Gambia. He had taken appropriate precautions and so asked how the infection could have occurred. The doctor used a diagram of the organism’s life cycle to explain this, including the stage of development at the time of transmission by the mosquito.
What stage in the organism’s life cycle did the doctor most likely highlight?
Other approaches (to be used sparingly)
If there seems no obvious clinical scenario for a science question, question writers may resort to scenarios that are clinically less relevant. These are not ideal because they are often simply a device to seek factual recall, with the scenario there almost for the sake of it. In the early undergraduate years, this approach can be justified as a way of accustoming medical students to finding answers to questions within a scenario, in preparation for subsequent undergraduate and postgraduate clinical examinations. However, in postgraduate examinations, it is more important for the clinical context to be obviously relevant to the question.
Thus, the methods below might appear more a way of sticking to the brief of including a scenario in each question while testing mere knowledge, rather than testing understanding and the application of knowledge.
Clinical teaching scenario
A 26-year-old woman was being ventilated on the intensive care unit. Her investigations showed…. (full blood count, urea and electrolytes, and arterial blood gases). The intensivist asked the registrar why the patient’s oxygen dissociation curve had shifted to the left.
What factor is most likely to have influenced this shift?
This is still a good question as it preserves the single-best-answer format and requires candidates to apply their knowledge to the scenario.
Consultation with a scientifically-informed person
A 34-year-old woman developed progressive external ophthalmoplegia caused by a mitochondrial cytopathy. She had a scientific background and asked the doctor to explain the function of mitochondria.
What deficiency of their normal function is most likely?
This question just about maintains a single-best-answer format but could appear contrived.
Student teaching scenario
During a lecture on cell metabolism, the presenter’s summary slide listed the key functions of the mitochondrial respiratory chain complex 1.
What function is most likely to appear first on this list?
Although this involves a scenario it is a less good question, involving only factual recall with no application of knowledge to a specific situation.
Conclusion
The large majority of science knowledge can be assessed in examinations using single-best-answer questions with realistic clinical scenarios, although these often require considerable thought and preparation by question writers. The crucial element of a scenario for a science question is that it must be essential to the question and not redundant. Writing a clinical scenario that is based on everyday clinical practice and includes material essential to answer the question is the best way to ensure that the knowledge being assessed is clinically relevant.
Competing interests
PS is associate medical director for quality for the MRCP(UK) and is sub-dean for assessments, Institute of Medical Education, Cardiff University School of Medicine.
Acknowledgements
We thank Derek Waller and Chris McManus for their helpful input into the paper.
References
- 1.Case S, Swanson D. Item writing manual: Constructing written test questions for the basic and clinical sciences, 3rd edition (revised). Philadelphia, PA: National Board of Medical Examiners, 2004. [Google Scholar]
- 2.Membership of the Royal Colleges of Physicians of the United Kingdom Annual Review 2015. London: MRCP; (UK: ), 2015. Available online at www.mrcpuk.org/sites/default/files/documents/MRCP%28UK%29-annual-review-2015.pdf [Accessed 19 November 2015]. [Google Scholar]
- 3.Case SM, Swanson DB, Becker DF. Verbosity, window dressing, and red herrings: Do they make a better test item? Acad Med 1996;71:S28–S30. [DOI] [PubMed] [Google Scholar]