Developing a personalised, evidence-based and inclusive learning (PEBIL) model of blended learning: A cross-sectional survey

Abstract

Whilst the use of various blended learning models preceded the COVID-19 pandemic, the abrupt shift to remote delivery served as catalyst within the sector in enhancing digital solutions to meet immediate student needs. As we emerge from the pandemic, a return to purely didactic and impersonal in-person teaching seems anticlimactic, with the return to the lecture theatre seeing many lecturers trialling various digital tools in creating more interactive in-person, synchronous, and asynchronous sessions. In evaluating students’ experiences of the various tools and approaches applied by academic staff, a survey was developed by a multidisciplinary team of educators at Cardiff University’s School of Medicine exploring student perceptions of e-learning resources (ELRs), as well as student experiences of various blended learning approaches. The primary aim of this study was to evaluate student experience, satisfaction, and engagement with ELRs and blended learning. A total of 179 students (undergraduate and postgraduate) completed the survey. 97% confirmed that e-learning resources were blended within the teaching they received, with 77% rating the quality of e-learning as good-to-excellent and 66% reporting a preference for asynchronous resources that enable them to learn at their own pace. A variety of platforms, tools, and approaches were identified by students as meeting their diverse learning needs. We therefore propose a personalised, evidence-based and inclusive learning (PEBIL) model enabling the application of digital technologies both on and offline.

Introduction

Engaging students as active participants in their learning is undeniably one of the key tenets of modern pedagogic practice (e.g., Gopinathan et al., 2022; Khan et al., 2017; Moreno & Mayer, 2000). Whilst the use of various digital (online) technologies in facilitating the delivery of engaging and interactive teaching preceded the need for remote delivery imposed by successive COVID-19 lockdowns, the pandemic accelerated the diversification of teaching practices and required the rapid upskilling of lecturers’ technological capabilities, out of necessity. The lifting of restrictions and the ‘return to in-person teaching’ on face-to-face courses has been met with initial enthusiasm but has also led to many questions being asked about the future use and focus on digital learning methods and where we go from here (Rapanta et al., 2021). Returning to the purely didactic lecture, whether in-person or online, may be an effective way of disseminating considerable amounts of information, consistently, in a short space of time, but deprives students of learning at their own pace at a time of their choosing and positions them simply as passengers in their own learning experience, passively participating in the activities (e.g., Langegård et al., 2021).

What do we mean by blended learning?

The term ‘blended learning’ has been used in several different settings to refer to a number of different modes of delivery (Alammary et al., 2014). The various models of blended learning have been applied to traditionally face-to-face programme, for instance through the use of a flipped-classroom approach, as well as to online courses, such as through the use of an enriched-virtual model (Hrastinski, 2019). The early post-pandemic period has resulted in a further broadening of the definition of ‘blended’ in this context, making it challenging at times for students (as well as staff) to understand exactly what is meant by the term. Those teaching in the Higher Education sector have, however, most likely been engaging in some form of blended learning before the pandemic struck. For instance, medicine and other healthcare curricula have largely embraced student-centred curricula that involve case-based or problem-based learning (e.g., Thistlethwaite et al., 2012). Such curricula have ensured a variety of learning experiences for students are interwoven within the delivery of teaching content (e.g., Hassoulas et al., 2017). In essence, the blend of teaching activities and learning opportunities has existed for quite some time within the sector, however the application of specific digital tools and platforms in further embedding students in their learning had been somewhat lacking.

Catering for diverse learning, and teaching, needs

It is well-established that blended learning models are perceived by students as providing greater flexibility, thereby enhancing their learning experiences through further engagement and interactivity (Garrison & Kanuka, 2004; Nortvig et al., 2018). Students have, however, also reported that the further time commitment required to engage with numerous activities (if not timetabled appropriately) and dealing with some sophisticated technologies couples with a lack of social interaction may prove challenging at times (Langegård et al., 2021; Vaughan, 2007). Furthermore, whilst a flexible approach to content delivery enables an inclusive approach by targeting numerous learning styles, inclusivity also needs to be considered from the perspective of the student having adequate access and knowledge of how to use specific digital platforms (e.g., Castro, 2019; Navarro et al., 2016; Rasmitadila et al., 2020). Similarly, an important consideration is ensuring that staff are adequately supported ‘in-house’ in using various tools that may enhance the delivery of their material by actively engaging students in the sessions. In parallel with this, the Higher Education Institutions themselves must be able to provide adequate and sustained infrastructures that effectively support digital learning resource design, development, deployment and maintenance.

Bruggeman et al., (2021) identified key attributes that may influence whether lecturers implement blended learning. Pedagogic beliefs, knowledge and understanding of the pedagogy underpinning blended learning, confidence and self-reflective competence, as well as openness to trying something new all reportedly influenced staff engagement with blended learning models. Whilst winning the hearts and minds of certain colleagues may present an early challenge in some instances, transformative self-reflection is required of educators in not merely considering what we do and why, but what we could do (Biggs & Tang, 2011).

A driver for post-pandemic innovation

The literature on blended learning illustrates that various models can be used to improve attendance, engagement, performance, and the overall student learning experience in the context of both in-person as well as online teaching (e.g., López-Pérez et al., 2011; Rasmitadila et al., 2020; Vaughan, 2007; Wright, 2017). As such, it is clear that a targeted approach is required to explore (a) the learning needs of respective student cohorts (e.g., Biber & Heidorn, 2021), (b) accessibility and inclusivity considerations (for students and staff), (c) time commitment in developing and completing (from the student perspective) asynchronous and/or synchronous content (e.g., Moorhouse & Wong, 2022), and (d) the pedagogic basis for the blended lesson plan (e.g., Iqbal et al., 2021). Consideration must therefore be given to colleagues’ pedagogic and digital competencies and barriers they may face (Hinojo-Lucena et al., 2019). Furthermore, a review by Pettersson (2018) reported that future research on teaching staff digital competency should focus on organisational structures, the introduction of frameworks that aim to close the gap between policy and practice, as well as developing new approaches that focus on enhancing digital competencies.

In considering how best to meet the diverse learning needs of our students, we created the multidisciplinary School of Medicine Digital Education Group that was tasked with overseeing the provision of digital medical education and staff training. Members of the group include a school-wide network of academics, clinical academics, student representatives, administrative staff, and research staff who deliver teaching. The introduction of this group was deemed necessary in ensuring that organisational structures were in place to identify and subsequently meet the learning needs of students and digital needs of staff (Pettersson, 2018).

The group was tasked with exploring students’ experience and perception of e-learning. Data was collated on a range of synchronous and asynchronous teaching activities across undergraduate and postgraduate programmes of study in the school of medicine. The primary aim of the current study was therefore to determine what platforms and interactive activities enhanced the student learning experience, and the contexts in which this was best facilitated. Specifically, this aimed to build on primary research projects that previously investigated students’ reported experiences of blended learning on both undergraduate and postgraduate courses (e.g., Pellas & Kazanidis, 2015). In an effort to identify a model of blended learning that was tailored to the needs of our student cohorts, it was considered crucial that the student voice shape any such approach, undertaking a targeted approach in attempting to effectively closing the gap between policy and practice within this specific cohort of diverse students.

Materials and methods

Design and materials

A cross-section survey was undertaken, in keeping with published guidance on best practice in conducting survey research (Kelley et al., 2003). Ethical approval was sought and provided from the Cardiff University School of Medicine Research Ethics Committee (SoMREC). A mixed methods research design was employed, with the electronic survey consisting of ten Likert scale items and four free-text items. All items were designed in consultation with academic and clinical staff, administrative colleagues, educationalists, learning technologists, and student representatives. In addition, the inclusion of specific themes within the survey were influenced by prior work reported by Pellas and Kazanidis (2015), Langegård et al. (2021) and Nortvig et al. (2018), who identified key factors involved in student engagement and satisfaction with e-learning as well as blended model of learning. Specifically, the survey focused on student satisfaction with e-learning resources (ELRs) made available on their respective courses, engagement with different types of e-resources and which were preferred by students, and how ELRs were integrated into curricula and teaching provision. Items were also adapted from a prior survey conducted by Hassoulas et al. (2017), who looked at a blended approach in designing case-based teaching content and activities.

Participants

An electronic survey was deployed via the university’s virtual learning environment (VLE), Blackboard, to students enrolled on undergraduate and postgraduate programmes of study within the School of Medicine. Specifically, a call for participants was sent as an announcement using our school’s VLE, which also generated an email that went out to all student users enrolled on undergraduate and postgraduate modules offered by the school of medicine. This was done in order to recruit students from as many programmes as possible and, therefore, to provide a more representative picture of students’ experiences with ELRs and various models of blended learning being adopted by programme teams without the focus being on solely on a single programme of study. As such, a voluntary response sample was recruited from our undergraduate programmes of study (of which the MBBCh medical programme is by far the largest in student number) and postgraduate taught programmes within the school of medicine. Participants received information about the study at the onset, with consent required to proceed to the survey items. Upon completion of the electronic survey, a debrief section was made available to participants providing further information about the purpose of the study.

Data Analysis

A mixed methods approach was adopted, with quantitative data collated in the form of Likert scale items and qualitative data collated from free-text items on the survey. Descriptive data was primarily captured from the responses to the Likert scale items. Content analysis was used to analyse the free-text responses, as this type of qualitative analysis has been identified as focusing on external validity and being suited to qualitative healthcare education research (Downe-Wamboldt, 1992).

Results

A total of 179 (157 undergraduate and 22 postgraduate) students participated and completed all fourteen items included in the survey (see Table 1). Of the undergraduate respondents, 148 were enrolled on the MBBCh (medicine) programme, with 9 undergraduate students enrolled on various BSc programmes provided by the school of medicine (e.g., medical pharmacology, intercalated degree in medical education). The 22 postgraduate students who participated in this study were enrolled on taught as well as research programmes.

Table 1.

Survey respondent distribution by type of degree

		Undergraduate		Postgraduate
	Total	MBBCh	Other	Taught course	Research
Type of student	100% (179)	83% (148)	58% (9)	8% (14)	4% (8)

Engagement with ELRs

Of the participants, the vast majority (97%), reported that they had the opportunity to engage with e-learning resources as part of their programme of study (see Table 2). Of those who responded that they did not have the opportunity to engage with ELRs (3%), two-thirds were postgraduate research students and one-third BSc students.

Table 2.

Survey responses highlighted the availability of, and preference for, ELRs on programmes of study

	Yes	No
As part of your course (or postgraduate research project), have you had the opportunity to engage with any e-Learning Resources (ELRs)?	97% (173)	3% (6)
Would you like to see more ELRs associated with your course (or postgraduate research project)?	61% (109)	39% (70)

The majority of ELRs that were made available to students (see Fig. 1) were created by staff (academic, clinical academic, and learning technologists) and were either provided as mandatory or optional learning resources (see Fig. 1A). In addition, 57% of students reported that the ELRs made available to them, either as mandatory or optional supplementary e-learning, encouraged active participation in their learning (see Fig. 1B). This included the use of interactive features such as questions embedded at various points within the resource, the use of engaging case studies, and the application of prior knowledge to various clinical scenarios. Examples of passive forms of ELRs quoted by students included pre-recorded lectures without interactive features embedded within these.

Fig. 1.

Type of e-learning resources available and type of engagement, as reported by survey respondents. (A) Type of e-learning resources regarding mandatory or optional nature, as well as if developed “in house” or externally. (B) Type of engagement in the e-learning resources used by participants (passive, active or other). (C) Rating effectiveness of e-learning resources used by survey respondents. The average rating was 3.99. (D) Synchronous nature of the e-learning resources used by survey respondents. Note: ‘Mandatory’ denotes an ELR that required satisfactory proof of completion and was a ‘timetabled activity’ for students; ‘Optional’ denotes an ELR that was constructively aligned with curricular Learning Outcomes, but students could choose to engage or not (and as such, was not formally timetabled). Such optional ELRs were considered as ‘adjunctive learning aids’

ELR popularity and preferences

In considering whether students showed a preference for either synchronous or asynchronous resources, 66% reported a preference for the latter (see Fig. 1D). In addition, 77% of respondents rated the ELRs on the respective programmes as effective and useful to their learning (see Fig. 1C), whilst 20% rated the ELRs as neither particular effective or ineffective.

More specifically, 81% of students enrolled on the MBBCh programme rated their ELRs as good-to-excellent. Of the remaining undergraduate students enrolled on BSc courses such as Medical Pharmacology and Medical Education, 67% of respondents rated the ELRs on their programmes as good-to-excellent. Postgraduate students, however, who took part in this study painted a slightly different picture to that of their undergraduate counterparts. Only 43% of postgraduate students who completed the survey rated the ELRs they were presented with as good-to-excellent in terms of effectiveness and usefulness to learning, with 33% rating these as neither particular effective or ineffective. In addition, 19% reported that their courses did not make use of any ELRs as part of the learning they had undertaken (Table 3).

Table 3.

Rating of ELR effectiveness and usefulness as a proportion of respondents (%)

	Good-to-excellent (%)	Neither good nor poor (%)	Poor (%)	N/A No ELRs (%)
MBBCh	81	16	3	0
BSc course	67	11	11	11
Postgraduate courses	43	33	5	19

Open-ended follow-up questions were included in the questionnaire to capture more granular feedback from students regarding their ratings of the ELRs. Content analysis was used to identify key themes within the qualitative data, as well as to determine the frequency with which these themes appeared. Tables 4 and 5 illustrate the key themes identified by students on the basis of how they rated the ELRs.

Table 4.

Content analysis frequency table of key themes identified by respondents rating the ELRs as good-to-excellent in effectiveness and usefulness

	Interactive, Engaging, Active learning	Content: concise, fills gaps	Learn at own pace	Consolidate learning, revision	Inclusive, ease of access	Visuals are appealing	Better than lectures
Total Frequency (%)	35	20	17	14	5	5	4
MBBCh	32	19	15	13	5	5	4
BSc	2	-	1	1	-	-	-
Postgrad	3	1	1	-	-	-	-

Table 5.

Content analysis frequency table of key themes identified by respondents rating the ELRs as neither good nor poor in effectiveness and usefulness

	Variation in quality of ELRs	Length and complexity	Lacking formal teaching or in-person option	Aesthetics and appeal	Difficult to motivate, concentrate (cognitive considerations)
Frequency (%)	50	17	13	10	10
MBBCh	43	17	10	-	10
BSc	-	-	-	3	-
Postgrad	7	-	3	7	-

Whilst the number of MBBCh respondents far outnumbered the BSc and postgraduate respondents to the survey, our findings indicate that all cohorts of students who completed the survey agreed on four key themes that make for good-to-excellent ELRs. Namely that these must be interactive, engaging and encourage active learning; they enable students to learn at their own pace; they present content in a concise manner; and they enable the consolidation of information (see Table 4).

In total, 19% of respondents rated the ELRs provided on their course as being neither effective nor ineffective. Whilst five key themes were identified across cohorts in the school, there were slight differences in the justification for the ratings awarded between respondents from the MBBCh, BSc, and postgraduate programmes. For example, those enrolled on the MBBCh programme highlighted that there was considerable variation in the quality of the ELRs, and that certain resources were too lengthy/time consuming and complex. They also highlighted that, in certain instances, the ELRs were not accompanied by any form of more formal teaching or instruction, and they found it challenging to motivate themselves to focus on completing the standalone ELRs. The postgraduate students also emphasised the issue of variation in quality of the ELRs they were provided, and that in some instances no formal instruction or support accompanied the resources. Interestingly, aesthetics and visual appeal appeared to be a common theme identified by BSc and postgraduate students, whereas this was not commented upon by MBBCh students.

Only 3% of students who completed the survey rated the ELRs on their respective programmes of study as being poor in effectiveness and usefulness. Nevertheless, it was deemed crucial to further explore the responses from these students in ascertaining why the rating was awarded and where improvements can be made. Four key themes were identified across courses: variation in the quality of the ELRs; lack of interactive and/or engaging content; length and complexity; and issues with content (difficult to follow or out-of-date). Only one theme emerged, however, in our postgraduate respondents and that was the variation specifically in the quality of ELRs. Similarly, the BSc students focused on only one of these four themes, which was the lack of interactive and engaging features of certain ELRs. This highlights that course-specific considerations are as important as overarching policy considerations when formulating a school-wide strategy on digital education and blended learning.

Of note, when students were asked whether they would like more or fewer ELRs on their respective courses, 61% responded that they would prefer more. Specifically, 64% of MBBCh respondents, 67% BSc, and 41% PG would prefer more ELRs. The low number of postgraduate students requesting additional ELRs is likely due to some of these programmes being ‘distance learning’, online courses that already include a large number of ELRs. When asked what type of ELRs students prefer and would like to see more of, a variety of different kinds of e-resources were highlighted. The most popular included short e-modules providing concise overviews of content covered, ELRs that provided the opportunity to practice exam-type questions, and short animations. Respondents also reportedly wanted to see additional interactive virtual patient cases and podcast-type pre-recorded talks (see Fig. 2).

Fig. 2.

Proportion of preferred ELRs reported by students (responses expressed as percentages)

Considering the technologies used to facilitate inclusive learning

Respondents to the survey across programmes were also requested to indicate what electronic device(s) they used to access the ELRs (see Fig. 3A), as well as what web browsers they made use of during engagement (see Fig. 3B). Most students reportedly use laptops or iPads to access ELR material, with a minority using smartphones or desktop computers. In terms of browsers used to access the ELRs, the majority of students reported that Goggle Chrome was by far their preferred browser, followed by Safari. These findings illustrate the importance of considering the compatibility of ELRs in relation to devices used by students and the browsers relied upon. Beyond these, there are further considerations that are required in ensuring that no students are disadvantaged by the use of certain digital platforms hosting ELRs.

Fig. 3.

Type of technology used by students to access e-learning resources. (A) Type of device and (B) browser used to access e-learning resources. Note: some ELRs directed students to use digital devices with ‘large screens’ due to the visual nature of the content. In addition, where there were known browser issues in displaying / running some ELR content, students were directed to use specific browsers that the ELR creators knew would display the content correctly

Discussion

Overview of results

Our findings reveal that the inclusion of asynchronous, interactive ELRs in teaching delivery enhance the student learning experience on the whole. Students also report to access external e-resources in further aiding their learning, where these aren’t available as in-house developed e-learning. This highlights the importance of a comprehensive blended learning and digital education strategy, as ensuring the provision of a range of in-house learning resources will also provide staff with some reassurance that key supplementary material being accessed by students aligns with programme-specific higher level learning outcomes and assessments. In addition, many external ELRs require a subscription, which can prove very costly for students. In relation to external subscriptions, one student requested that additional in-house ELRs are made available to further supplement learning, and commented that.

“in the last year I stopped my subscription because it is quite expensive and couldn't really justify paying for it, but it is probably the most effective learning”

Qualitative feedback from students revealed that the popularity of ELRs across cohorts was mostly due to the interactive and engaging nature of these resources, which also helped in the consolidation of knowledge by addressing multiple learning needs. The ability to learn at one's own pace was another popular theme that emerged from the student free-text feedback. There were also more specific comments made by students as to why they identified ELRs as effective tools that enhance their learning experience. For instance, one student commented that,

“ELR’s really useful! I enjoy the fact you can do them in your own time and go back over them, I find lectures quite difficult to keep up with”.

Having the opportunity to also explore interactive content in a way that facilitates knowledge consolidation and fill gaps identified in prior knowledge was another common theme that emerged in the data across student cohorts. One student in particular commented that,

“active ELRs (are) a productive way of learning and assessing my own knowledge and finding gaps in my knowledge and filling those gaps”.

The use of ELRs in creating interactive virtual patient cases has also proven a popular application of such a blended approach among students, with one medical student stating that,

“most ELRs are valuable methods of engaging with clinical information and teaching, as they can be conducted at our own pace and promote active engagement with the content”.

Whilst the majority of undergraduate and postgraduate students surveyed identified ELRs as beneficial to their learning, those who did not provide a high rating on the effectiveness of ELRs identified the main problem as being the variation in experience of ELRs on their respective programmes of study. Specifically, a lack of interaction, engagement, and unappealing delivery of content were the main reasons provided by students who did not feel certain ELRs contributed to the learning experience. For instance, one student commented that,

“the information in the ELRs is often crucial, but they could be made more visually appealing and tidy which would aid learning”.

This highlights the need to ensure that academic and clinical staff are sufficiently trained and equipped in the creation of interactive, high quality ELRs that serve as effective supplementary learning materials for students across cohorts.

Other common themes identified by students in terms of how ELRs could be improved include ensuring that content is presented in concise, bite-sized chunks of information (as opposed to being too lengthy and complex) and that ELRs are not used to replace all in-person teaching. This latter point is crucial, as it highlights that students explicitly request teaching be delivered using a range of methods in meeting their diverse learning needs. ELRs do appear to have an important role to play in the future of higher education, but the appropriate application thereof is key. Similarly, in-person lectures should embrace the use of technology and include interactive elements, moving beyond the purely didactic approaches of the past.

Proposing a new blended model in response to student needs

As digital technologies become increasingly embedded within teaching delivery in Further and Higher Education, identifying the most appropriate tools and platforms for specific lesson plans can present new challenges, but crucially also, new opportunities to enhance the student experience. As such, the pedagogy and technology will need to be guided by key course considerations and the evidence gathered as to ELR efficacy amongst degree programme cohorts. Here, priority is to ensure activities align with Learning Outcomes (LOs) and assessments (Biggs & Tang, 2011), but are also inclusive, enabling and allow students to engage as active participants in their learning.

In our proposed model, in selecting the blend of activities that align with LOs and assessments, three main requirements should be considered: (i) if the lesson plan is tailored and personalised to the specific subject and cohort of students, (ii) if the proposed plan is supported by the literature/evidence, and (iii) if it ensures that no students will be disadvantaged (fully inclusive). Hence, our ‘PEBIL’ (Personalised, Evidence-Based and Inclusive Learning) model focuses primarily on these three considerations in the creation and tailoring of lesson plans.

Through the introduction of the PEBIL model and lesson plan matrix, further consideration can be given to key stages within lesson plans, namely pre-event engagement, the type of event (e.g., lecture, training, workshop), and type of post-event activity. The matrix serves as a guide that applies the PEBIL model to ensuring that relevant tools, based on sound pedagogy, are considered when putting together a lesson plan. Figure 4 presents a flowchart demonstrating the application of the PEBIL model in the designing and planning of lesson plans.

Fig. 4.

Flowchart depicted the application of the PEBIL (Personalised, Evidence-Based, and Inclusive Learning) model of blended learning to lesson planning

In further supporting staff delivering synchronous as well as asynchronous teaching, a simple lesson plan matrix has also been proposed to aid in identifying the most appropriate platforms and technologies that can be used in the delivery of a specific session (as well as pre- and post-session activities). The suggestions are guided not only by the evidence, but also by student feedback in terms of what has worked well in the past (Figure 5).

Fig. 5.

Matrix of platforms and/or digital technologies that can be used to host pre and post session activities, as well as technologies that can be incorporated within in-person or remotely delivered teaching sessions

Our School of Medicine Digital Education Group (acronym ‘HIVE’- Hybrid, Interactive & Virtual Environments) therefore strives to support staff in developing a universally high standard of ELRs. Furthermore, our group has not only facilitated the introduction of the PEBIL model but has embedded the student voice as a guiding principle in the additional support that has been made available to staff in the School of Medicine and the wider university (e.g., workshops, peer mentorships and support).

Summary

As educators, we should ensure that we are sensitive to the diverse ways in which students learn and assimilate taught content. Equally, we should be mindful that our teaching staff are every bit as diverse, and this should be factored into the design and planning of all educational content creation. The delivery of teaching during the pandemic highlighted the diverse learning needs of our students, as well as the challenges that colleagues face in adapting to new methods of teaching provision. Our model has addressed key themes, and has aimed to fill gaps, identified in the literature. This includes developing a tailored approach to meeting students’ unique learning needs (Biber & Heidorn, 2021), as well as adopting a balanced approach to incorporating asynchronous and synchronous components of a lesson plan to ensure the student learning experience is enhanced without adding to their timetabled workload (e.g., Moorhouse & Wong, 2022). Furthermore, the PEBIL model is unique in that it emphasises the importance of both student and staff literacy when it comes to digitally enhanced learning on the physical campus, as well as in the virtual classroom. As such, our PEBIL model equips educators with the tools and the confidence to deliver their content in a way that takes advantage of their strengths, and in doing so, further enhances the student learning experience. Our team has begun developing staff training modules, to be delivered in a hybrid and blended manner, with the aim of addressing gaps in knowledge and digital teaching provision. We have implemented the PEBIL model in designing the content and the teaching activities within these modules, demonstrating the model’s relevance in continuing professional development as well.

Data availability

The datasets generated and analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Disclosure statement

The authors report no conflicts of interest (financial or otherwise). The authors alone are responsible for the content and writing of this article.