Medical education research remains the poor relation

Abstract

Research into medical education is stagnating and urgently needs the resources to become more rigorous and relevant say Mathew Todres, Anne Stephenson, and Roger Jones

The requirement that clinical practice should be based on the best available evidence has been paralleled by calls for medical education to become more evidence based.^{1 2 3} This has resulted, among other initiatives, in the establishment of the Best Evidence for Medical Education (BEME) Collaboration⁴ and the Campbell Collaboration, an off-shoot of the Cochrane Collaboration. The BEME initiative includes dissemination of best evidence to support medical education and the encouragement of a culture capable of nurturing more rigorous and better funded research.

Evidence from the United States suggests such nurturing is much needed. In 2004, Carline analysed reports of medical education research in two major North American journals (Academic Medicine and Teaching and Learning in Medicine) and found that only a minority of studies were supported by external research grants.³ She was critical about the quality, rigour, and generalisability of most of these studies. Her concerns were echoed last year by Chen and colleagues,⁵ who advocated moving the focus of medical education research from learners to patient oriented clinical outcomes, thus increasing the relevance and its likely attractiveness to funders. A review of 290 medical education studies published during 2002 and 2003 found that only one quarter had received external funding; the median amount of funding obtained was $15 000 (£7700; €11 500) with an interquartile range of $5000 to $66 500.⁶ Private foundations, as opposed to federal institutions, were the most common source of these research grants.

Recent medical education research

We were unable to find any recent information about the state of published medical education research in the UK and Europe. We therefore reviewed research published in 2004 and 2005 in two general medical journals, the BMJ and Lancet, and the two leading medical education journals, Medical Education and Medical Teacher. We included only primary research papers and secondary research studies (systematic reviews and meta-analyses).

During 2004 and 2005, none of the 390 research papers published in the Lancet was in the field of medical education. Only 11 of the 399 papers published in the BMJ related to medical education. We combined these with the research papers in Medical Education (207) and Medical Teacher (169) for further analysis.

Research topics, methods, and funding

Most of the research papers used observational, cross sectional survey designs, and less than 10% reported longitudinal cohort and before and after studies (table). Of the 210 (54%) studies that used questionnaires, 178 (85%) provided no details of the validation of survey instruments. Very few papers reported studies using experimental designs, with case-control studies and randomised controlled trials each accounting for less than 3% of the sample. We did not find any meta-analyses.

Table 1.

 Details of medical education research published in three journals during 2004-5

	No (%) of studies (n=387)
Study design
Observational, cross sectional	267 (69)
Longitudinal cohort	31 (8)
Before and after studies	31 (8)
Other	58 (15)
Focus of research
Undergraduate medical curriculum	240 (62)
Continuing medical education	85 (22)
Postgraduate medical training	43 (11)
Other	19 (5)

The research focused predominantly on the undergraduate medical curriculum, with the remaining papers dealing mostly with continuing medical education and postgraduate medical training. Table 2 shows the range of research topics. Student examinations and assessment, curriculum design, professional development, learner characteristics, and teaching methods accounted for 70% of studies. We found little research on potentially important topics such as selecting students for medical school (12 studies), patient issues in medical education (eight), and career choices for medical graduates (four). Most of the research was conducted in the UK, continental Europe, and North America.

Table 2.

 Topics for reported studies in medical education, 2004-5

Research topics	No (%) of studies (n=387)
Assessment or examinations	64 (17)
Curriculum design	62 (16)
Professional development	56 (14)
Learner or student characteristics	52 (13)
Teaching methods	46 (12)
Technology in medical education	23 (6)
Teacher development	22 (6)
Other	62 (16)

The research was largely unfunded. Only 30% (116) of the papers stated that the study had external funding; 47 had internal funding and 224 gave no information on funding, which we assumed meant that there was none. Less than half of these studies (170) were collaborative ventures between two or more institutions. Collaborative studies were more likely to receive external funding than non-collaborative studies (53% v 47%; χ²=4.44, P<0.05), and papers published in Medical Education were more likely to be externally funded than those in Medical Teacher (39% v 20%; χ²=17.85, P<0.001). We were unable to identify any other associations between publication type, study design, and funding source.

Randomised controlled trials

We identified 10 randomised controlled trials.^{7 8 9 10 11 12 13 14 15 16} Most were published in Medical Education and most concerned undergraduate medical education. These studies largely reported comparisons between different methods of teaching and used a range of assessments and end points. Many of them fell short of the criteria developed by the International Committee of Medical Journal Editors for reporting randomised controlled trials. Several did not include a clear a priori hypothesis, accurately specified end points, or a sample size calculation. Type II statistical errors were common, and CONSORT diagrams indicating the flow of subjects in the control and experimental groups were generally absent.

Problems for research

This review suggests that research into medical education has not moved on, in terms of funding and methods, over the past five to ten years despite repeated initiatives to inject greater methodological rigour and to find better ways of funding studies.

Whether the absence of medical education research from the Lancet and the small number of studies published in the BMJ are causes for concern is debatable. It might be argued that the research should appear in specialist medical education research journals and be read by medical educationalists, but research on topics such as examinations, student selection, predictors of academic success, and professional trajectories in medicine is of wider relevance and deserves larger audiences. The publications in Medical Education and Medical Teacher reflect slow progress in terms of developing methods and attracting funding, with observational designs and unfunded studies still being the rule.

The research landscape in medical education is reminiscent of primary care and health services research 20 years ago, when we lacked a cadre of trained researchers, used primitive research methods, and struggled for funding. The fortunes of applied medical researchers began to improve when funders began to understand the importance of a firm evidence base for clinical care and the delivery of services. It is unclear whether the funders of medical education (the Department for Education and Skills, through the Higher Education Funding Councils) or its providers (the universities and medical schools) are convinced of the importance of a carefully built evidence base to ensure best practice. Even when medical education research is funded, the source of funding is often a charity or foundation rather than a publicly supported funding council. Although several large UK medical research charities provide funding for educational training fellowships, only the Arthritis Research Campaign supports fellowships in medical education research.

The situation is made more difficult by the lack of appreciation of research into medical education in our medical schools. One of the effects of the research assessment exercise (used to allocate research grants in the UK) has been to prioritise the laboratory based sciences over behavioural and applied sciences and research over education, with commensurate shifts in staffing, funding, and academic kudos. This has further disabled medical educators, who are now working harder than ever to keep up, often in the context of high student numbers and increasing bureaucratic complexity. One positive development, however, is that the 2008 assessment exercise will include medical education research.

The medical education research community needs to mobilise support for its mission to provide the best education for medical students. This means thinking of imaginative ways to create a critical mass of educational researchers so that cross-centre, inter-institutional, and multidisciplinary studies can be conducted. Such studies are more likely to produce generalisable results than current research. When these studies can be directly linked to meaningful outcomes, such as the quality of care provided by qualified doctors, they will become more attractive to funders. Critical topics in medical education, such as criteria for selecting medical students, predictors of success and failure at medical school, the development of a mature professional outlook and of personal and interpersonal skills appropriate to various branches of medicine, are all amenable to high quality research.

Undergraduate and postgraduate curriculum reforms over the past 20 years have resulted in major changes in the way that our students and postgraduates are taught, often on the basis of nothing more than pragmatism, fashion, and whim. Not all of these have been good for medical education or for the doctors that it produces—unwieldy student numbers, poorly developed web-based learning introduced as a substitute for direct personal contact, and an obsession with vertical and horizontal integration that led to the destruction of valuable free-standing courses in subjects such as pathology and therapeutics. We must ensure future changes will be beneficial.

Summary points

• Medical education research lacks methodological rigour and support from funding councils

• Most studies focus on the undergraduate medical curriculum

• Important topics such as patient issues in medical education, selection of students, and career choices are under-researched

• Multicentre studies using good experimental designs are needed