Is evidence based medicine neglected by royal college examinations? A descriptive study of their syllabuses

Although the value of evidence based medicine has been debated,¹ the benefits of teaching it to undergraduates² and postgraduates³ have been shown and have been acknowledged in the development of the new undergraduate medical curriculum.⁴ However, lack of adequate training is a major obstacle to postgraduates.³

Evidence based medicine has four distinct steps⁵: formulate clear clinical questions from a patient's problem; search the literature for relevant articles; evaluate the evidence for its validity and usefulness; apply useful findings in clinical practice.

This report examines the level of skills in evidence based medicine that are formally assessed by the royal colleges in the United Kingdom, through a review of the colleges' syllabuses for postgraduate examinations that are compulsory for specialist training.

Methods and results

We reviewed all syllabuses in effect on 20 October 1999 that were held by 16 faculties or royal colleges in the United Kingdom, representing 15 major specialties. Surgical examinations (held by three royal colleges) were reviewed separately. Subspecialties in surgery and pathology were grouped as a single specialty. Radiology and oncology, which are both examined by the Royal College of Radiologists, were analysed separately.

We reviewed each syllabus to determine whether the skills required for the four steps in evidence based medicine were assessed. The authors initially reviewed the syllabuses independently, and any disagreements (of which there were very few) were resolved by discussion. Where it was clear from the syllabus that a specific section of the examination is dedicated to examining skills in evidence based medicine, we obtained sample or past papers if available.

These skills were not substantially assessed in seven of a total of 17 syllabuses, for five out of 15 major specialties represented: general medicine, surgery, paediatrics, ophthalmology, and radiology. (We did not regard assessment of basic statistics alone as adequate for step three.) A dedicated section of the examination explicitly assesses these skills in five syllabuses (table). Most of these five syllabuses focus on the candidates' ability to evaluate the evidence for validity, but emphasis varies on the skills in evaluating the evidence for its usefulness, formulating a clear clinical question, searching for relevant literature, and implementing useful findings in clinical practice. In the remaining five syllabuses in which skills in evidence based medicine are mentioned (see tables on BMJ website), there are no dedicated procedures for examining these skills in anaesthetics, obstetrics and gynaecology, and oncology; in pathology and occupational health, candidates have to submit a dissertation with original research data.

Comments

One third of the specialties do not assess skills in evidence based medicine in their examination system. Examinations often exert a steering effect on the curriculum, and it would be difficult for future doctors to keep their professional knowledge and skills up to date unless these skills are learnt during training and are regularly applied in clinical practice.

Two thirds of the postgraduate examinations have no dedicated sections for the examination of skills in evidence based medicine. Even in syllabuses where these skills are examined, this is generally limited to skills in evaluating the evidence for validity. However, to practise evidence based medicine effectively, skills in formulating a clear question, evaluating evidence for its usefulness, and applying findings to clinical practice are equally important.²

We acknowledge that royal college examinations are only one element of specialist training and that revision of some syllabuses may be under consideration. Nevertheless, the strategies of the royal colleges for assessing skills in evidence based medicine should be reviewed, possibly by following good examples of the few that assess these skills in some depth.

Table.

Examinations with one or more dedicated sections for examining elements of skills in evidence based medicine explicitly

Specialty	Examination	Format of examination	Duration of examination in evidence based medicine	Contents	Steps in skills in evidence based medicine examined*
Family planning	MFFP	Dissertation		Special area of study (research or audit)	1, 3, 4
		Critical reading paper	90 minutes	Questions are designed to test ability to evaluate published material, to analyse and use data, to base clinical decisions on published scientific evidence, and to apply epidemiological findings to clinical working practice
General practice	MRCGP	Critical reading paper	90 minutes	Questions designed to test:	1, 3, 4
				• knowledge and interpretation of general practice literature
				• ability to evaluate and interpret written material presented (including taking an approach of evidence based medicine to a clinical scenario)
				• ability to integrate and apply theoretical knowledge and professional values
Public health	MFPHM	Critical appraisal	120 minutes	Commentary and application of material in an article from a journal	3, 4
		Dissertation		Showing competence in:
				• critical literature review
				• assessment of effectiveness and efficiency of an aspect of health service	1, 2, 3, 4
Psychiatry	MRCPsych	Critical review paper	90 minutes	Questions on an abridged paper to test:	3, 4
				• appraise the article for validity
				• appraise the article for clinical usefulness
				• application of findings to clinical practice
Accident and emergency medicine	FFAEM	Clinical topic review	NA	To submit two clinical topic reviews, each of about 2000-2500 words, to test the ability to:	2, 3, 4
				• conduct a literature review
				• critically appraise the research material found
				• write a review article that enhances the reader's understanding of the topic
		Oral discussion of published paper		Understand research methodology
				Assess soundness and clinical relevance of research work	3, 4

^*The steps in evidence based medicine are: 1 Formulate clear clinical questions from a patient's problem. 2 Search the literature for relevant articles. 3 Evaluate the evidence for its validity and usefulness. 4 Apply useful findings in clinical practice.