Increasing medical students' engagement in public health: case studies illustrating the potential role of online learning

Abstract

Background

The value of e-learning in medical education is widely recognized but there is little evidence of its value in teaching medical students about public health. Such evidence is needed because medical students' engagement with public health has been low. We present three recent case studies from UK medical schools to illustrate diverse ways in which online approaches can increase medical students' engagement with learning public health.

Methods

A comparative case study approach was used applying quantitative and qualitative data to examine engagement in terms of uptake/use amongst eligible students, acceptability and perceived effectiveness using an analytic framework based on Seven Principles of Effective Teaching.

Results

Across the three case studies, most (67–85%) eligible students accessed online materials, and rated them more favourably than live lectures. Students particularly valued opportunities to use e-learning flexibly in terms of time and place. Online technologies offered new ways to consolidate learning of key public health concepts. Although students found contributing to online discussions challenging, it provided opportunities for students to explore concepts in depth and enabled students that were uncomfortable speaking in face-to-face discussions to participate.

Conclusions

E-learning can be applied in diverse ways that increase medical student engagement with public health teaching.

Introduction

Public health understanding, knowledge and skills are essential to the practice of clinical medicine and to the health of the population.^1,2 ‘Tomorrow's Doctors’, the UK General Medical Council's guidance on the knowledge, skills and behaviours required by undergraduate medical students, states that students should be able to ‘apply to medical practice the principles, method and knowledge of population health and the improvement of health and healthcare’ and be able to ‘discuss from a global perspective the determinants of health and disease and variations in healthcare delivery and medical practice’.³ Despite this, however, public health can be perceived by medical students as irrelevant and unnecessary, a perception reinforced by it being given lower priority in schools within an increasingly crowded medical curriculum.⁴

As students embrace online technologies in general, there is increasing commitment to using online methods of learning in medical education.^5,6 Whilst there is evidence that online methods can be as effective as face-to-face teaching to medical students and other health professionals,^7–9 there is still little evidence of how to use these methods successfully.¹⁰ Until recently, most medical schools used online formats as static repositories for teaching materials rather than as active learning resources, offering little opportunity for generating evidence around the range of interactive online approaches available.⁴ As a result, there is little to guide public health educators as to which online approaches may best meet their objectives of engaging medical students to learn core public health skills needed to practise clinical medicine.

Aim

We present three recent case studies from UK medical schools, each covering a different aspect of public health, to illustrate the diverse ways in which online approaches can be used to increase medical students' engagement with learning public health. We define engagement in terms of three dimensions: uptake/use amongst eligible students, acceptability and perceived effectiveness.

Methods

Design

This study used a comparative case study approach,¹¹ to identify similarities and contrasts in student uptake/use, acceptability and perceived effectiveness of online approaches in different contexts. Three examples of new models of teaching public health using purely online or blended (combined online and face-to-face) approaches were selected to illustrate ways in which students can be supported to learn public health principles. These comprised: More detail is given in Table 1 and in Supplementary data for CS2 and CS3.

Table 1.

Case study descriptions

	(1) Evidence-based practice: e-lectures	(2) Healthcare public health: online self-study module	(3) Pilot global health and communicable disease control: asynchronous discussions
Setting	University of Birmingham	UCL Medical School	Brighton and Sussex Medical School (BSMS)
Students	Year 3 of 5-year course (n = ∼370)	Year 4 of 6-year course (n = ∼400)	Year 4 of 5-year course (10 students volunteered to pilot the course)
Subject	Evidence-based medicine and research methods: covering study designs, interpreting data, critical appraisal, developing research questions and clinical guidelines	Principles and practice of population screening (screening policy, test characteristics, harms and benefits, evaluation)	Global health protection focussing on comparing communicable disease control in UK with impact and practice in low-income settings
Previous format and rationale for change	Five sessions, each involving a 1 h lecture, 1 h of self-directed learning and a 2 h face-to-face small group tutorial	One-hour lecture, followed by a 1 h face-to-face small group tutorial, delivered four times	A mapping exercise in 2012 revealed that the BSMS curriculum was not meeting recommended global health competences
	79% of evaluation comments on lectures were negative and student attendance at lectures was poor (<50%). Students suggested e-lectures as an alternative	Lecture feedback was mixed. Some students complained they had learnt material previously but some still had a limited grasp of basic concepts. Students suggested online formats	While a small number of core global health sessions were introduced, there were areas not covered in these sessions which are of value particularly to students with a global health interest
e-Learning approach	Live lectures were replaced by e-lectures, made available to all Year 3 students via the university's virtual learning environment. Students continued to be timetabled for a 1 h lecture, 1 h of self-directed learning and a 2 h face-to-face tutorial	Live lectures were replaced by a short online module made available to all Year 4 students 2 weeks before face-to-face teaching	A pilot module on global health and communicable disease control was developed in collaboration with People's-Uni, a charity which provides low-cost online public health education in low-income countries (www.peoples-uni.org)
	Lecturers recorded the e-lectures using PowerPoint and headphones. The lectures were similar in format to the live lectures, with activities adapted from the live lecture	Module design was informed by Mayer's principles of effective multimedia learning.12 It comprised short lecture casts, video clips, multiple-choice questions and links to external resources, with short ‘diagnostic’ quiz for students to self-assess prior knowledge and decide where to focus	The module comprised a 2-week online discussion facilitated by a tutor, simulating a ‘virtual classroom’ focussed on realistic scenarios, e.g. measuring the impact of HIV in a community, management of a measles outbreak in rural Uganda
	Questions could be raised on-line or directly with tutors in the face-to-face teaching	Students and trainee doctors created content, tested pilot versions and provided feedback	Discussions were asynchronous, rather than real time so participants' posts remain visible for the duration of the module and others can respond hours or days later. Tutors comprised three People's-uni alumni, all health professionals in low-income settings—Swaziland, Ethiopia and Papua New Guinea
		Questions could be raised online or directly with tutors in face-to-face teaching
Evaluation data	Student feedback survey: Likert scale and free text questions	Student feedback survey: Likert scale and free text questions	Focus group amongst participating students
		Site usage monitoring data	Site usage monitoring data

• – Case study 1 (CS1): Evidence-based practice—conversion of existing face-to-face lectures to e-lectures for ∼370 students (University of Birmingham MBChB).

• – Case study 2 (CS2): Healthcare public health—conversion of lecture material to an interactive, multimedia module, developed with student involvement, for ∼400 students (UCL Medical School MBBS).

• – Case study 3 (CS3): Global health protection—pilot of a new, optional module for a small group of students (∼10), employing asynchronous online discussions across several countries (Brighton & Sussex Medical School BM BS).

Data collection

A mixture of quantitative and qualitative data were used to evaluate CS1 and CS2; CS3 primarily utilized qualitative data (Table 1).

Analysis

To assess uptake/use, self-report (CS1) and website monitoring data (CS2 and CS3) were used to generate the percentage of eligible students that accessed each resource.

To assess acceptability, in CS1 and CS2, we compared student feedback on e-learning modules with data captured the previous year, when students received face-to-face lectures presenting similar materials. Responses were dichotomized to capture the proportion of students giving the resources high scores. We assessed the statistical significance of the difference between years using a chi-squared test.

To examine perceived effectiveness of e-learning approaches, we conducted a thematic analysis of data from free text responses in CS1 and CS2 and coded focus group data from CS3, combining inductive and deductive approaches to identify learning generalizable to all three cases.¹¹ We drew on Chickering and Gamson's Seven Principles of Effective Teaching¹³ (Table 2), based on evidence that increased engagement is likely to be a proxy for learning as an analytic focus to assess effectiveness.

Table 2.

Chickering and Gamson's Seven Principles of Effective Teaching (as described by Chickering and Ehrmann¹³)

Principle	Explanation
1. Good practice encourages interaction between students and faculty.	Frequent student–faculty contact in and out of class is a most important factor in student motivation and involvement. Faculty concern helps students get through rough times and keep on working. It also enhances students' intellectual commitment and encourages them to think about their own values and plans.
2. Good practice encourages interaction and collaboration between students.	Learning is enhanced when it is more like a team effort than a solo race. Good learning, like good work, is collaborative and social, not competitive and isolated. Working with others often increases involvement in learning. Sharing one's ideas and responding to others improves thinking and deepens understanding.
3. Good practice uses active learning techniques.	Learning is not a spectator sport. Students do not learn much just sitting in classes listening to teachers, memorizing pre-packaged assignments, and spitting out answers. They must talk about what they are learning, write reflectively about it, relate it to past experiences, and apply it to their daily lives. They must make what the learn part of themselves.
4. Good practice gives prompt feedback.	Knowing what you know and do not know focuses your learning. In getting started, students need help in assessing their existing knowledge and competence. Then, in classes, students need frequent opportunities to perform and receive feedback on their performance. At various points during college, and at its end, students need chances to reflect on what they have learned, what they still need to know, and how they might assess themselves.
5. Good practice emphasizes time on task.	Time plus energy equals learning. Learning to use one's time well is critical for students and professionals alike. Allocating realistic amounts of time means effective learning for students and effective teaching for faculty.
6. Good practice communicates high expectations.	Expect more and you will get it. High expectations are important for everyone–for the poorly prepared, for those unwilling to exert themselves, and for the bright and well-motivated. Expecting students to perform well becomes a self-fulfilling prophecy.
7. Good practice respects diversity—talents, experience and ways of learning.	Many roads lead to learning. Different students bring different talents and styles to college. Brilliant students in a seminar might be all thumbs in a lab or studio; students rich in hands-on experience may not do so well with theory. Students need opportunities to show their talents and learn in ways that work for them. Then they can be pushed to learn in new ways that do not come so easily.

Results

We set out below the ways in which e-learning approaches engaged students in learning public health in three dimensions: uptake, acceptability and perceived effectiveness (using The Seven Principles of Effective Teaching).

Uptake amongst eligible students

Across the case studies, 67–85% eligible students accessed the online resources. In CS1, 85% of students reported they accessed the online material. Site monitoring data from CS2 showed that 67% of students accessed the module before face-to-face teaching. Their access appeared to be prompted by a reminder email (Fig. 1). Multiple-choice questions were most highly accessed, by ∼75% of those that enrolled. In contrast, 70% viewed content pages about screening policy and 55% videos on pages about doctors' experiences of screening. In CS3, where students volunteered to participate in a module that was not a core part of the curriculum, uptake was similar with 80% (8/10) eligible students posting at least two discussion forum contributions over the 2-week course.

Fig. 1.

CS2: student uptake of online learning in the 24 h before small group teaching.

Acceptability of online teaching compared with lectures

In CS1 and CS2, where online learning replaced lectures, students rated online learning more favourably than lectures. Satisfaction with small group teaching remained unchanged (Table 3).

Table 3.

Student ratings for CS1 and CS2: lectures versus online

	Student feedback questions	High scores (%) (‘agree’ or ‘agree strongly’ or scoring 4/5 or 5/5)
		Lecture	Online	Percentage difference	P
CS1	The lectures (2013/2014)/e-lectures (2014/2015) helped me to learn	40.6	67.3	26.7	<0.0001
	The small group tutorials helped me to learn	62.5	63.5	1.0	0.88
n		381	368
CS2	How useful was the lecture (2013)/online module (2014)?	31.9	57.8	25.9	<0.0001
	How useful was the small group teaching?	47.0	53.5	6.5	0.051
n		166	147

In CS3, students reported they found online discussion ‘helpful’ and they ‘really enjoyed it’. However, some reported feeling ‘pressured’ because it was ‘much more difficult than I thought it was going to be’.

Perceived effectiveness

Principle 1: interaction between students and faculty

There was no suggestion that students in any of the case studies felt ‘short changed’ by less face-to-face contact with tutors. In fact, some CS2 students proposed that more face-to-face screening teaching should be delivered online. In CS1 and CS2, students did not use the optional online discussion forums to interact with faculty or other students. In contrast, in CS3, the global health module, students clearly valued the opportunity to interact with tutors working as healthcare workers overseas:

I think the advantage to online is that you can have people from different places all at once. [CS3, focus group]

It was just nice to hear from someone with first-hand experience, like you could really just relate to that a bit more, I think, than reading it, you know, in a textbook [CS3, focus group]

Principle 2: interaction and collaboration between students

CS3 indicates that some students were intimidated by the requirement to interact with other students in the discussion forum. They compared this interaction with other types of discussion, e.g. real-time online, face-to-face interactions with friends or anonymous discussions. The asynchronous nature of discussions with peers also appeared to intensify their need to write more, and more carefully:

You're thinking, ‘Oh what I can add?’ and then another email's come through, and you think ‘Oh gosh’ and I've got to reference all of this, and by which time you've made 500 words of something and try and copy and paste it into things, make sure the spelling's all right. It made into like a real … like a big thing. [CS3, focus group]

You always felt you had to sort of match the level of the posts before. [CS3, focus group]

CS3 students' experiences may provide some insight into why students in CS1 and CS2 opted not to use discussion forums. However, even though only three CS1 students took up the opportunity to post questions, 42 students reported that they found the forum useful. The CS3 focus group data illustrate how students may have benefited from observing the discussion:

My flatmate … didn't post anything, but we were talking about it a lot and she was reading everything, but she never got around to actually writing anything. [CS3, focus group]

Principle 3: use of active learning techniques

The degree of active learning (i.e. where students are required to do something other than simply listening, reading or watching) in the case studies varied. In CS1, the evidence-based medicine module, e-lectures contained limited opportunities for active learning, although tutors invited students to pause and think about a question before proceeding with the video up to three times per lecture. Despite this, CS1 students described in their survey responses various ways in which they engaged with the e-lecture content. For example, they paused the video to research their queries online ‘in real time rather than afterwards when I've had chance to forget’. CS2, the screening module, provided active learning opportunities primarily through multiple-choice questions. These were the most accessed part of the site, and students judged them ‘really helpful’. In CS3, the focus group comments illustrate how the discussion forum prompted deeper learning than more passive forms of delivery:

I think I would have learnt a lot less if it was just a big long passage that was in like a paper, a review paper that I'd just read, kind of thing. Because it was a discussion it was a bit more dynamic, I think I learnt more. [CS3, focus group]

Principle 4: provision of prompt feedback

As discussed in the previous theme, the automated feedback from quizzes was valued during online learning (CS2), and a minority of students used the opportunity to improve their performance by trying quizzes repeatedly until they got them right.

Principle 5: emphasis of time on task

All three approaches sought to give students freedom over how long they spent learning. In CS1, students reported they spent 87 min on average (range: 10–330 min) watching an 85-min lecture and clearly valued the opportunity to pace themselves, primarily to engage with concepts they found challenging:

I prefer e-lectures, as they allow you to take the lecture at your own pace. You can replay sections, which you didn't quite understand the first time, pause it when you need a break, and skip sections you feel you already know [CS1, survey]

In CS2, students spent less time than they would have done in a lecture (median time = 37 min) with durations ranging from less than 10 min to over 3 h. While they valued the opportunity ‘to study at our own pace’, they primarily wanted to skip concepts ‘we have already covered’.

In CS3, several students reported spending much longer on the module than they expected and found it a strain to have constant access:

It was always there, it was something that I could do at any time so I felt guilty if I wasn't looking at it or wasn't working. But with face-to-face, you know that's going to happen at that time, so you prepare for it. [CS3, focus group]

Principle 6: communicating high expectations

Theme 6 is omitted because the learning outcomes which set expectations were unchanged between traditional and online delivery modes.

Principle 7: respects diversity—talents, experience and ways of learning

Students in all cases studies valued the flexibility over how and when they learned, and the opportunity to go back to materials. This mode of learning was also particularly suitable for students who found traditional lectures or speaking in face-to-face classroom settings challenging:

In a normal lecture I don't have time to pause and think about concepts and type them out to consolidate my learning. I really struggle in normal lectures so I found e lectures where I could play and pause as I wish extremely useful. [CS1, survey]

You can embarrass yourself more face-to-face whereas online, even with the discussion, I felt like it was like I was kind of safe behind my computer and if I put a weird answer out there, that it would maybe be discussed in a very rational way. [CS3, focus group]

Discussion

Main findings

These three case studies illustrate how online approaches, combined with face-to-face teaching, can engage medical students in learning public health. Most (67–85%) eligible students accessed materials, and rated them more favourably than live lectures. Students particularly valued the opportunity to use e-learning flexibly in terms of time and place. They also valued opportunities to consolidate their learning, e.g. by doing quizzes, researching queries in real time, and contributing to online discussions. Whilst several found it ‘really hard’ to construct posts, the requirement to post online meant they read materials more attentively.

What is already known on this topic

There is a recognized dearth of evidence about the success of any method of public health teaching in medical schools.¹⁴ As stated in the ‘Introduction’ section, medical students can be more disengaged with public health teaching than many other subjects; hence, our specific need to seek different learning strategies. Our experience as educators indicates that, more importantly, public health may be particularly well suited to the inclusion of online methods that enable working alone, in a self-paced way. First, critical appraisal, often taught within public health in medical school curricula, requires close reading of complex papers and hence is best suited to environments which maximize concentration and reflection. Secondly, generating screening characteristics and risk measures requires some students to practise examples and work through calculations in much more detail than others. In time-limited, face-to-face sessions, those that instantly grasp these skills find it frustrating to be ‘kept back’, whilst those who struggle with numerical skills find it stressful to expose their difficulties in front of fellow students.

Ben-Shlomo also recognizes the importance of bringing public health teaching out of the lecture theatre into real life.¹⁴ As demonstrated in CS3, students highly valued the capacity of online technology to bring them in direct communication with overseas professionals with relevant first-hand experience.

What this study adds

The diversity of the three case studies was a strength in illustrating a range of ways in which online learning can be applied. The similarities in uptake across all three cases suggest that well-designed e-learning modules are likely to reach the majority of eligible students. The mixed-method approach ensured breadth from the quantitative data on a large sample of students in CS1 and CS2, complemented by depth from the CS3 focus group data. The focus group suggested that students were as concerned about peer observation as tutor feedback. This prompted us to consider this as a possible explanation for why few students participated in online discussion in CS1 and CS2. However, just as Beaudoin has indicated, ‘lurkers’ (i.e. individuals that observe but do not contribute to online discussions) in this study benefitted from online discussions even if they had not participated.¹⁵

At 67–85%, uptake across our case studies was higher than reported in previous studies, e.g. Grant et al. reported 37.5% of medical students used multimedia evidence-based medicine self-study modules.¹⁶ While encouraging, this may reflect the increasing use and familiarity with e-learning platforms and social media. Our finding that students rated e-learning higher than lectures is consistent with Awad et al.'s evaluation of a public health e-learning package for medical students and George et al.'s review of e-learning for health professionals where in both cases, satisfaction was higher with online than traditional approaches.^7,17

A key theme across all our case studies was the capacity of online approaches to overcome time constraints. Students in CS1 and CS2 valued the opportunity to skip material they knew or to spend longer to consolidate difficult concepts. In CS3, using online asynchronous discussions enabled staff to offer additional Global Health teaching to motivate students without negotiating space in an already crowded timetable and also enabled health professionals working across different time zones to participate. However, the combination of peer pressure and unrestricted access to email postings provided by Smartphones led to some students feeling pressured and recognizing the maturity required to manage this method of learning effectively. These advantages and problems with asynchronous discussions are recognized and studies point to the need for innovative facilitation approaches from tutors to generate productive discussion.¹⁸

Given the dearth of literature on this subject, we also offer more details for others who are wishing to develop materials on what the switch to online learning modes added over and above face-to-face teaching in CS2 and CS3 (Supplementary data) and reflections on developing online learning materials on public health (Box 1).

Box 1. Creating successful online public health learning resources: things we have learned.

We offer some insight from experience in developing and evaluating these case studies about how online approaches can best be applied to increase future doctors' engagement in learning public health:

1. E-learning is not cost-free¹⁹ but resources do not need to be a barrier to generating high quality materials

   • Signpost to selected public health resources already available on university websites and on YouTube rather than generating content from scratch.
   • Consider recording e-lectures—they require comparatively little extra effort from tutors and when they are designed well, they can successfully engage students.
   • Recruit recently graduated students to inform design of resources/develop materials—they have the best idea of what students need and where they may struggle.
   • To maximize the effectiveness of discussion forums, prepare students in this method of learning and ensure that tutors are trained in online facilitation.
   • Do be prepared to invest time in developing and updating online materials, but they can be re-used so upfront ‘investment’ may be recouped in later years.
2. Tailor online approaches to the subject, and learning goals.

   • Use multiple-choice questions with automated feedback where expectations of students are comparatively clear to enable students to practise core skills (e.g. manipulating data to calculate risk ratios, screening test characteristics) or test their understanding.
   • Online discussion fora can be useful ways for students to explore and research areas of public health in more depth, particularly where the evidence base is less clear.
3. Balance autonomy with giving direction

   • Offer students flexibility in when and how they access e-learning by making resources accessible on tablet, phone and computer.
   • Use timely email/text or social media reminders to prompt students to access resources.

Limitations of the study

It is not possible to conclusively attribute changes in student satisfaction between lectures and online formats to this change in format. However, in CS1 and CS2, students' satisfaction scores were similar in both years for small group sessions. These followed lectures and online delivery and comprised the same content and format each year. With no major changes to curricula or admission processes, there was no reason to expect the student cohorts had changed significantly from year to year either.

There were fundamental differences in the nature of the case studies which limited their comparability. CS1 and CS2 aligned to models of provider-generated content (referred to by Ehlers amongst others as e-learning ‘1.0’), whilst CS3 aligned to models of user-generated content (e-learning ‘2.0’). This shift from ‘1.0’ to ‘2.0’ e-learning brings different considerations for evaluation.²⁰ In addition, CS1 and CS2 applied to all students, whilst CS3 was for a small number of self-selected students. Also, fostering interest and enthusiasm in students is not a core part of The Seven Principles framework, so it may not adequately capture the extent to which online learning addressed this big challenge for public health teaching in medical schools.⁴

Conclusions and further research

E-learning is in line with societal trends²⁰ and its value to medical education has been recognized.²¹ However, the position of public health in the medical curriculum is often insecure⁴ and as Ben-Schlomo comments, there is a need for an evidence base to protect this teaching from ‘whims and fashions’.¹⁴ By illustrating diverse ways in which e-learning can be successfully used to engage medical students in public health, this paper contributes to the evidence base. Our findings also suggest a need for further studies that explore how e-learning can best contribute to equipping all medical students with sufficient public health skills and understanding to practise medicine in any specialty effectively.

Supplementary data

Supplementary data are available here.

Funding

The UCL work was supported by E-Learning Environments who provided advice and mentoring throughout the project and an E-Learning Development Grant (ELDG 2013/2014) to fund student involvement in the development of the module. The BSMS Global health module was supported by a Learning and Teaching fellowship from the Centre for Learning and Teaching at the University of Brighton. Authors H.B. and J.Sh. are funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Ethics

CS3, the BSMS Global health module, was granted Research Governance Approval from the BSMS Research Governance and Ethics Committee (RGEC ref: 14/007/JON). CS1 and CS2 did not require ethical approval.