Cost-effective undergraduate medical education?

Introduction

The undergraduate medical curriculum is in a labile state, with rapid developments in evidence-based interventions and a shift in emphasis ‘from gaining knowledge to a learning process that includes the ability to evaluate data as well as to develop skills to interact with patients and colleagues’.¹ In addition Government-led initiatives to widen participation to medical schools and increasing student numbers² mean that medical education programmes need to develop alternative learning and teaching resources and strategies to ensure that students from increasingly diverse backgrounds with different learning needs are able to achieve core outcomes. In this editorial we advocate evidence-based curriculum development that also addresses constraints on time and resources: cost-effective curriculum development.

Why should we undertake cost-effectiveness analyses of new pedagogical approaches?

In the UK the General Medical Council state that ‘teaching and learning systems must take account of modern educational theory and research, and make use of modern technologies where evidence shows that these are effective’.¹ Recent developments in medical education approaches and tools to aid learning and teaching offer medical educators new opportunities in curriculum delivery. In recent years there has been a shift from didactic teaching to a student-centred approach with the aim of encouraging deep learning. But the ascendency of expressions such as ‘deep learning’ epitomizes the challenge. How do we ensure that our students don’t simply memorize facts but develop understanding and are able to relate ideas and concepts? Educators have been encouraged to adopt a range of approaches including: problem-based learning, virtual learning environments, simulated patients and alternative assessment strategies such as the OSCE, some of which may incur additional costs compared to a more traditional approach. But are such changes evidence-based and are we ensuring that students meet core learning objectives in an efficient way?³

One may argue that only through formal assessment of both the incremental costs and effectiveness of the new compared to the standard education approach can we undertake efficient curriculum development. However, while evidence-based medicine is now common place, evidence-based medical education is less established.⁴ Baernstein et al. recently demonstrated an increase over the last four decades in the number of publications assessing educational intervention for undergraduate medical students.⁵ However the authors state that ‘considerable opportunities for improvement remain’. It appears that the robust methods used to provide evidence of efficacy of new medical treatments are not routinely used to assess medical education despite efforts by the Best Evidence Medical Education collaboration.⁶ Methods to guide policy including assessment of the incremental cost-effectiveness of different approaches are vanishingly rare, with only five studies on undergraduate education of those published between April 2006 and March 2007 even reporting costs.⁵

Assessing the cost-effectiveness of medical education

Which costs?

The cost of implementing a new teaching approach will vary based on perspective. From a societal view we may argue that graduate entry courses are cost-inefficient given the additional resources and training involved. But the tantalising question remains: what are the benefits to society of this additional education?⁷

A more accessible approach is to capture the costs accrued from the perspective of the Medical School or College which will include: curriculum development, preparation, delivery and assessment costs, and may reflect educator costs and physical inputs such as the learning environment and materials.⁸ This is analogous to the approach used by organizations such as the National Institute for Health and Clinical Excellence (NICE) which appraises new interventions on the basis of their costs to the healthcare system rather than their costs to society as a whole.⁹

Although it is appealing to capture costs as a monetary value, costs are incurred in both student and educator time which face fixed limits. Here the concept of ‘opportunity costs’ is key, since it identifies the costs of an action in terms of the alternatives forgone as a consequence. With increasing time pressures on the curriculum this is an important consideration.

What outcomes?

In an academic environment new education approaches may be rapidly adopted, but with a full curriculum and so many opportunities for changes how do we know what will be the most effective strategy and how do we define effectiveness? The ultimate aim of medical education is to improve healthcare delivery and the wellbeing of patients (levels 4a and 4b of the modified form of Kirkpatrick's hierarchy of levels of evaluation for an educational intervention as developed by Freeth et al. in 2002) (Table 1).¹⁰ Assessing the impact of a change in the undergraduate curriculum on patient care may prove difficult, but not impossible. For example, competency-based outcomes may be assessed using simulated or real patients particularly for final-year medical students. It is relatively straightforward to gather evidence of student views on the learning experience but this surrogate measure may not reflect how the education impacts on clinical practice. It is our belief that we should strive to obtain robust data on the higher levels of the hierarchy.

Table 1.

Mode of outcomes of interprofessional education. Adapted from Freeth et al.¹⁰

1 Reaction	Learners' views on the learning experience and its interprofessional nature
2a Modification of attitudes/perceptions	Changes in reciprocal attitudes or perceptions between participant groups. Changes in perception or attitude towards the value and/or use of team approaches to caring for patients
2b Acquisition of knowledge/skills	Including knowledge and skills linked to interprofessional collaboration
3 Behavioural change	Identifies individuals' transfer of interprofessional learning to their practice setting and changed professional practice
4a Change in organizational practice	Wider changes in the organization and delivery of healthcare
4b Benefits to patients	Improvements in health or wellbeing of patients

Assessing cost effectiveness: a place for randomized controlled trials (RCTs)?

Undergraduate medical students provide us with a large cohort in which to assess the effectiveness of education. Despite this, RCTs of undergraduate medical education appear rare.⁵ Through randomizing students between different educational strategies we ensure that differences observed in outcome between the groups may be attributable only to the different strategies used or to the play of chance. Statistics allows us to examine the plausibility that observed differences may be attributable to chance alone, and if this is not plausible, the only alternative is that the observed effect is due to the intervention. Randomized trials can thus enable us to obtain robust estimates of the impact of an education initiative on the attainment of outcomes and avoid potential bias and confounding that may arise from cohort or case-control studies. Trial protocols can be designed to allow us to collect the costs associated with different education approaches and to undertake within trial cost-effectiveness analyses. Care must be taken not to disadvantage students randomized to different interventions; this may be achieved through the use of crossover trials or more simply by offering the alternative approach in a limited form following the end of the trial.

Prognostic models based on trial results can also potentially be used to identify which student characteristics predict those who benefit more (or less) from different education approaches. These positivist analytical approaches may be complemented by interpretative investigation so as to understand why particular students identified from such models are under-performing with the aim of providing additional support or to aid the development of new programmes. Together these approaches could prove an invaluable tool in developing learning and teaching strategies and allow us to support our students' increasingly diverse needs.

Conclusion

Identifying which education approach is the most efficient use of medical school resources and in which ‘type’ of students is an important consideration for curriculum development but evidence from robust studies is sparse.^4,5 The costs associated with the implementation of new strategies in money and time and the incremental cost-effectiveness of implementing a new approach must not be forgotten in a constrained environment. How we choose to measure the cost-effectiveness of such interventions is open to debate. We teach our students the importance of evidence-based medicine yet the underpinning techniques including systematic reviews, RCTs, meta-analysis, and prognostic and economic models are rarely used to provide evidence for education development. History has shown us how belief in new treatments may be misplaced. Should we not practice what we preach and try to maximize patient benefit and improve the student learning experience in an efficient evidence-based way?

Footnotes

DECLARATIONS —

Competing interests None declared

Funding None

Ethical approval Not applicable

Guarantor MJC

Contributorship MJC conceived the idea and wrote the first draft of the manuscript. NF developed the manuscript. Both authors contributed to and approved the final version