Maxillofacial education in the time of COVID-19: the West Midlands experience

Abstract

COVID-19 has accelerated a reliance on virtual technology for the delivery of postgraduate surgical education. We sought to develop a regional teaching programme with robust quality assurance. Webinars were delivered on a weekly basis by subspecialty experts using Zoom™ augmented with interactive polling software. Trainee feedback comprised Likert item rating on content and delivery, free text comments and self-assessed confidence levels using visual analogue scale (VAS) scores. A focus group was also convened and transcripts assessed with grounded theory analysis. Likert items revealed 442 (93.2%) positive responses regarding content and 642 (96.7%) positive responses regarding trainer delivery. There were statistically significant improvements in VAS scores across all programme content. Key themes from the focus group analysis were the pragmatics of delivering online education, issues surrounding trainer interactivity in the virtual world, the identification of the FRCS as a driving factor and a desire for case-based content and pre-learning of information (the ‘flipped classroom’). We are continuing to be reactive to trainee feedback in developing our online learning programme which will also include a regional Moodle-based virtual learning environment (VLE), the subject of future educational research in our region.

Introduction

Technology-enhanced learning (TEL) has seen an increasing variety of formats in recent years, including virtual learning environments as either standalone interventions or as part of a ‘blended learning’ approach.¹ In the wake of the COVID-19 pandemic and associated rules surrounding social distancing to reduce the spread of Severe Acute Respiratory Syndrome coronavirus-2 (SARS-CoV-2), virtual education in postgraduate surgical training has seen something of an accelerated uptake.2, 3, 4, 5, 6

Whilst work has been done on virtual clinics following on from national guidance from the British Association of Oral and Maxillofacial Surgeons (BAOMS), there is a clear need to ensure trainees are not forgotten at a time when the quality of surgical training is under threat.7, 8, 9 We sought to therefore implement a robust online learning programme with inbuilt quality assurance from trainer and trainee feedback.

Methods

From the inception of the COVID-19 pandemic we put into place a weekly Zoom™ webinar with a nominated Consultant ‘screen sharing’ a presentation. Webinars often involved an additional “moderator” fielding questions from the ‘chat’ function. This was further augmented with interactive polling software such as Kahoot! and Socrative by MasteryConnect.

Trainees were asked to give feedback using a SurveyMonkey® questionnaire which examined perceptions of content and trainers on 5-point Likert items. Free text comments were available to express areas of particularly good practice and suggestions for improvements. Trainees were also asked to self-assess confidence levels using a visual analogue scale (VAS) from 0 to 100. Trainers were also asked to give formal feedback of their individual sessions.

In addition, trainees were asked to volunteer and representatives taken from each training unit to join a focus group using Zoom™. The focus group was facilitated by one of the authors (CF) in his capacity as an ‘external’ member (being a current Training Programme Director in another region) and used a semi-structured interview technique.

Analysis of the data included examining the proportion of respondents agreeing with positive statements on the Likert items along with Wilcoxon signed rank test of VAS scores. The transcript from the focus group meeting was analysed by two of the authors (RE and JG) using grounded theory analysis incorporating three levels of coding (open, axial and selective).10, 11, 12, 13

Results

Feedback surveys from seven Zoom™ webinar teaching sessions between 13th May 2020 and 24th June 2020 were analysed, with a total of 95 individual responses. Lecture subjects and trainee attendance is shown in Table 1 .

Table 1.

Number of attendees at each Zoom™ webinar.

Zoom™ webinar	Number of trainees attending
Facial reconstruction	15
Facial deformity: assessment	14
Vascular anomalies	14
Dental implants	10
Mandible trauma	13
Temporomandibular joint surgery	15
Melanoma	14

Table 2 demonstrates aggregate scores of Likert items with regards to content of the Zoom™ webinars. Responses were overwhelmingly positive with 442 (93.2%) responses agreeing with positive statements concerning content. Trainees particularly liked features such as polling software or quizzes and ‘pre-loading’ with information to prepare prior to the Zoom™ webinar. A common theme was the propensity to enjoy case-based discussions and ‘exam-style grilling’, with many trainees basing their perception of the value of the teaching on how well it might prepare them for the exit Fellowship examination. Trainees also liked it when two consultants delivered teaching, with one manning a ‘chat’ function and answering questions in tandem with the delivery of lecture-based content.

Table 2.

Composite responses to Likert item responses regarding quality of content across the teaching programme.

Likert item	Strongly disagree n (%)	Disagree n (%)	Neutral n (%)	Agree n (%)	Strongly agree n (%)
The learning outcomes were clearly stated at the outset	2 (2.1)	4 (4.2)	2 (2.1)	31 (32.6)	56 (58.9)
The webinar was pitched correctly for my learning needs	2 (2.1)	1 (1.1)	0 (0)	22 (23.2)	70 (73.7)
The content is directly relevant to my clinical practice	2 (2.1)	0 (0)	1 (1.1)	18 (18.9)	74 (77.9)
The webinar will change my clinical practice	2 (2.1)	0 (0)	8 (8.4)	30 (31.6)	55 (57.9)
The content of this webinar met my expectations	2 (2.1)	0 (0)	6 (6.4)	33 (35.1)	53 (56.4)

With regards to suggestions for improvements, trainees stressed the need to keep to the allotted time, a desire for recommended reading and clear references (where not provided), and a desire to be put ‘on the spot’, again driven by a clear wish for exam-style practice.

Combined scores of Likert items with regards to trainer performance is given in Table 3 and again, perception was positive, with 642 (96.7%) of responses agreeing with positive statements. Self-assessment of confidence levels on the VAS scales by respondents are shown by individual Zoom™ webinar in Table 4 , with a statistically significant improvement seen across the entire teaching programme.

Table 3.

Composite responses to Likert item responses regarding trainer across the teaching programme.

Likert item	Strongly disagree n (%)	Disagree n (%)	Neutral n (%)	Agree n (%)	Strongly agree n (%)
The lecture has inspired me to learn and explore the subject further	2 (2.1)	0 (0)	0 (0)	35 (36.8)	58 (61.1)
The lecturer was approachable	2 (2.1)	1 (1.1)	1 (1.1)	20 (21.1)	71 (74.7)
The lecturer ensured that the session was interactive	1 (1.1)	0 (0)	0 (0)	13 (13.8)	80 (85.1)
The lecturer used appropriate adjuncts e.g. diagrams, presentations through screen share	2 (2.1)	0 (0)	3 (3.2)	12 (12.6)	78 (82.1)
The lecturer ensured there was appropriate time for questions	2 (2.1)	0 (0)	0 (0)	20 (21.1)	73 (76.8)
The lecturer had good background knowledge of the subject	2 (2.1)	0 (0)	0 (0)	4 (4.2)	89 (93.7)
The lecturer was clear at all times through the webinar	2 (2.1)	0 (0)	4 (4.2)	20 (21.1)	69 (72.6)

Table 4.

Self-assessment scores as measured on VAS pre- and post-Zoom™ webinar by subject area. Statistical analysis used the Wilcoxon signed rank test and all improvements were statistically significant.

Zoom™ webinar	Mean (SD) VAS score (pre)	Mean (SD) VAS score (post)	p value
Facial reconstruction	59.9 (11.3)	72.5 (11.6)	0.0008
Facial deformity: assessment	55.5 (13.1)	74.2 (10.1)	0.00096
Vascular anomalies	50.8 (20.8)	75.1 (14.3)	0.00096
Dental implants	49.4 (13.7)	72.4 (8.7)	0.00512
Mandible trauma	66.8 (11.7)	80.1 (8.5)	0.00148
Temporomandibular joint surgery	56.3 (9.5)	74.3 (8.4)	0.00064
Melanoma	49.2 (13.3)	71.9 (10.5)	0.00148

Trainers felt that positive features included finding trainees had read up beforehand and that there was good interaction. Many found that add-ons such as Kahoot! helped with this interactivity and that the technology was dependable. Negative aspects included difficulty in gauging enjoyment using an online platform, as screen sharing may preclude seeing the faces of all attendees. One trainer described ‘feeling like one is on “transmit”’ and it was noted that some trainees kept their video cameras turned off which was ‘disconcerting’.

Focus group analysis

A comprehensive review of the three levels of coding derived from the focus group is demonstrated in Table 5 . A number of themes were identified.

Table 5.

Open, axial and selective codes derived from the grounded theory analysis of the trainee focus groups transcript.

Open coding	Axial coding	Selective coding
Good timetabling	Positive features of the logistics of course delivery	The pragmatics of delivering online learning
Well-structured
Preparation in advance
Regular
Spaced out
Maintaining momentum
Saves travelling
Predictability
Jump from topic to topic	Negative findings of the logistic/practical side of course delivery
Conflicts with other commitments
Loss of collegiality
Sound cuts out	Technical issues encountered
Miss what is happening
Interactivity	Good points of virtual interaction	The pros and cons of interacting with a trainer in the virtual world
‘Listen in’ on senior discussions
Recorded	Concerns raised about virtual interaction with trainers
‘Pulling faces’
Disconcerting
Lack of networking
Unable to see face
Learn without being watched
Disparaging faces
Problem with ‘being viewed’
Unable to see colleagues
Scored in front of everyone
Puts you on the spot	Exam preparation	The relationship between the content and delivery and the examination
Building up for the exam
Mapped to syllabus
Catered to the exam
‘Need to know’ for the exam
Short	Generic good points of the virtual teaching received by content	Overall teaching quality and areas of good practice or areas for improvement
Focused
Sharp
Honed down
Short and sweet
Consultant led
Going through cases	Areas of individual good practice experienced
Derived learning points from case
Discussing cases
Clear objectives
Having a moderator
‘Peripherally dipping’	Negative aspects of teaching experienced
Not suited to all topics
Information delivery only

The pragmatics of delivering online learning

The virtual teaching was well received which agreed with written feedback obtained from sessions. It was delivered at an appropriate pace and regularity and comments were made regarding the ease of access to teaching;

‘Saves travelling…’ (Trainee #1)

‘Easier than having an all day out….’ (Trainee #2)

One concern was regarding a lack of interaction with peers - this was seen in a negative light by some;

‘The most useful thing was meeting all the trainees and chatting, networking…’ (Trainee #2)

‘It is quite nice to see your colleagues and share the good, the bad and the ugly

and know where you are standing…’ (Trainee #1)

Interacting with a trainer in the virtual world

Interestingly the trainees had issues with interactivity with trainers, which mirrored those found by trainers in the written feedback. Some trainees preferred not to be watched and had video turned off. Several reasons were given for this including lack of access to cameras and personal preference. It was acknowledged that visual cues are an important facet in teaching. However, there were negative comments about facial expressions seen in some sessions;

‘…a couple of consultants that responded with disparaging faces’ (Trainee #3)

‘…I don’t think they know that they are making that face towards that

person.’ (Trainee #1)

Content delivery and the examination

There was considerable emphasis on teaching in relation to exam preparation. Trainees enjoyed the interactivity of the sessions which they felt mimicked the exam situation;

‘…it’s really good … for the exam…’ and ‘discussing cases to an exam level…

that’s useful’ (Trainee #5)

‘… puts you on the spot….that’s what we need in terms of building up for the

exam…’ (Trainee #6)

They also felt that exam preparation could be improved by focusing on exam technique and not just content;

‘… would like to see stuff more catered to part 1…there’s a technique to it and we need to cater to it.’ (Trainee #5)

Teaching quality and areas of good practice/for improvement

There were recurring themes in the discussion group regarding the quality of teaching. These included references to ‘short and sweet’, ‘focused’ and ‘sharp’ - highlighting that the virtual sessions helped keep interest and engagement in the session in comparison to didactic lecture based study days;

‘… a couple were bit heavy… useful but … information overload.’ (Trainee #6)

Trainees enjoyed sessions involving preparation, citing that they got more enjoyment from pre-reading material;

‘… best sessions where we had to do some preparation first…you get far more out of it.’ (Trainee #2)

‘… sessions that are interactive and require…preparation are the ones we get most out of’ (Trainee #6)

Case discussions were seen as a being valuable, improving interactivity and delivering clinically-based learning points. The focus group felt that consultant led delivery worked best;

‘teaching sessions of a trainee … it’s not the same standard as a consultant’ (Trainee #4)

‘they focus on the questions that you have always asked yourself…’ (Trainee #1).

Discussion

Maxillofacial surgery is no stranger to e-learning, with recent examples of TEL including the e-Learning for Healthcare (eLfH) initiative, e-FACE and the BAOMS resource MaxFax Bites.14, 15 In the COVID-19 era, the pandemic has threatened ‘conventional’ training opportunities but introduced a host of others, as well as accelerating the reliance of virtual technology for both the delivery of clinical care and medical education. A survey of American trainees found that 94.2% of residents had moved to web-based training and 96.5% reported modifications to training, many expressing anxiety about meeting programme completion requirements.¹⁶

Carlson highlighted the importance of Zoom™ in maxillofacial training during the pandemic, stressing the importance of audio and visual connection enabling interpersonal contact that reinforces engagement and deeper learning.¹⁷ Moe et al used a collaborative network to develop an inter-institutional e-learning programme using didactic lectures coupled with Q&A sessions with an expert panel, stressing that e-course development was iterative, particularly in the face of rapidly changing circumstances.¹⁸

The introduction of a virtual learning programme in our region has been well received, with consistently high ratings for content and teaching, as well as improvements in self-assessed knowledge levels. Trainees clearly liked the ease of access of the online webinars, regularity, concise format and interactive case-based webinars. There was a pronounced focus on acquiring the knowledge to pass the Fellowship examination, echoing worrying trends identified by others in surgical training.¹⁹ Encouragingly, pre-learning in preparation for an interactive teaching session of problem solving was actively sought, echoing successes previously with ‘flipped classrooms’ in our region.²⁰

Grounded theory is inductive and constructionist, acknowledging that conditions give rise to discursive practices and analysis with preconceptions fails to allow the data to speak for itself.11, 21 Perceived limitations of the virtual learning strategy were the loss of collegiality and networking. Difficult interactions with trainers in an ‘artificial’ encounter where communication may be strained, along with technical glitches and the inability to cover all topics were all points that need to be taken into account in planning the future. In higher surgical training, the transfer of theoretical knowledge in webinars can only represent part of an educational programme running in parallel with clinical work, with a clear need for this to be supplemented with more ‘hands-on’ teaching, such as simulation training and dissection.

Response bias may be felt to be an issue, with such a small number of easily identifiable trainees potentially feeling ‘pressured’ into responding positively to surveys and focus groups. We sought to mitigate this by the use of a independent facilitator from outside the region and ensuring that surveys and focus groups transcripts were anonymised. Self-assessed VAS scores are useful but previous work has highlighted the poor correlation of these with clinical performance, with potential over-estimation of clinical competence and increased risk-taking attitudes.22, 23

Future plans within the region will include continuing regular Zoom™ webinars in the post-COVID-19 era, combined with face-to-face study days aimed at more practical teaching events (e.g. cadaveric dissection courses). This will be augmented by a HEWM Moodle VLE (https://pgvle.co.uk/login/index.php), likely to be the subject of future educational research by our group. Above all, in shaping the future of our higher training in our region, we are mindful of being responsive to our trainees, for as de Cossart and Fish stress, ‘a curriculum has to be built carefully from proper foundations…...the nature of professional practice and the knowledge underpinning it are content specific and individual to a culture and its values.’¹⁹

Conflict of interest

We have no conflicts of interest.

Ethics statement/confirmation of patients’ permission

Not required. Participation of trainees in the focus group was entirely voluntary and all participants were informed of the intention to publish the findings and disseminate to a wider audience on a national/international platform. As such ethics approval was not required or sought.