Comparison of perceived educational value of an in-person versus virtual medical conference

Abstract

Purpose

Though prior literature has shown that virtual conferences improve accessibility and provide a comparable educational experience, further research is required to characterize their educational value.

Methods

In this repeated cross-sectional study, demographic and survey data were compared between attendance perspectives for the in-person student-led internal medicine conference held in 2019 and subsequent virtual conference held in 2020.

Results

There were 146 attendees at the in-person conference and 200 attendees at the online conference, in which 32 (22% response rate) and 52 responses (26% response rate) were gathered, respectively. Comparison of Likert Scale data via Mann-Whitney U Test revealed that learning objectives were better met in-person for the overall conference (p < 0.01) and didactic sessions (p < .05), but not for workshops, in which there was no significant difference. Survey takers noted the virtual conference to be more accessible on multiple factors, but felt as though their potential for interaction with other participants was more limited.

Conclusions

Results indicate that though the virtual conference appeared more accessible to attendees, overall learning objectives for the conference and didactic sessions were better met in-person. Interestingly however, there was no observed difference in perceived educational value for small group workshops.

Introduction

The coronavirus pandemic (COVID-19) has had a significant impact on the delivery of undergraduate medical education, causing the suspension of many areas of training.^1-3 Medical conferences have migrated virtually in adherence to social distancing guidelines, and though prior studies recognize this as an effective educational delivery method, there is a scarcity of research surrounding this topic.^4,5 Advantages include decreased costs, decreased carbon emissions, and increased accessibility, but may come at the expense of technical issues and lack of immersion.^6-11 Though some guidelines have emerged to address how to maximize the effectiveness of virtual meetings, few data are available on how participants perceive the educational value of online delivery in comparison to the traditional format.^10,11

The Book to Bedside (B2B) conference is an annual Internal Medicine conference hosted for clinical clerks at McMaster University that adapted to a virtual platform in 2020 using Zoom due to COVID-19.¹² By comparing the 2019 and 2020 conferences, this study aimed to investigate the differences in perceived educational value between virtual and in-person medical conferences, and characterize the advantages and disadvantages of the virtual platform. As one of the first large-scale Internal Medicine conferences for Canadian medical students to be adapted virtually, this study will serve to inform future practices in conference planning and medical education.

Methods

Study design

This repeated cross-sectional study compared survey results from the 2019 in-person conference and 2020 virtual conference. The study was compliant with the Personal Information Protection and Electronic Documents Act and approved with exemption by the Hamilton Integrated Research Ethics Board.

Recruitment

Advertising for both conferences began one month prior to the event and involved contacting the student governing bodies of each of the thirteen English-speaking Canadian medical schools. A registration fee of 35$, and 5$ were charged for registrants of the 2019 and 2020 conferences, respectively.

Survey design

An online survey was sent out to the 200 attendees of the 2019 conference and 146 attendees of the 2020 virtual conference. As no prior surveys with validity evidence were available, a custom survey was devised for the current study based on prior literature discussing the benefits and drawbacks of virtual conferences.^7-11 Demographic data, including age, year of study, and school of study were gathered. Likert scale data were gathered on attendees’ perspectives on medical conferences and to obtain feedback for each conference via scales ranging from 1 (strongly disagree) to 7 (strongly agree). The statement “my personal learning objectives for this session were met” were used to gauge attendee perception on the overall conference, didactic sessions, and workshop sessions.

Other features of the virtual conference were assessed on this same scale with statements surrounding financial, geographic, and scheduling accessibility, and perceived time of interaction with participants and presenters. Satisfaction of different aspects of the virtual conference were additionally assessed, including value for cost, audience engagement, satisfaction with Zoom as a virtual platform, and quality of technical support provided.

Statistical analysis

Analysis was done using Statistical Package for the Social Sciences (SPSS) 26.¹³ Mann-Whitney U Test was used to compare Likert scale data between virtual and in-person conferences as the data violated parametric assumptions.^14,15 Overall conference educational value was evaluated by comparing the Likert scale data on statements such as “My learning objectives for the conference were met” for the overall conferences, workshops, and large group sessions.

Results

Thirty-two and 56 responses were gathered for the in-person conference (22% response rate) and online virtual conference (26% response rate), respectively. Demographic information regarding level of study is summarized in Table 1. The in-person conference respondents consisted of 25 (80.6%) students from the host school, versus 22 (39.3%) in the virtual conference. Two-tailed Fisher Exact test revealed this to be a statistically significant difference of group composition between the two conferences (p < 0.001).

Table 1.

Demographic data of respondents of the in-person conference held in 2019 and virtual conference held in 2020

	2019 Conference	2020 Conference
Level of training
Pre-clerk Year 1	0 (0%)	5 (8.9%)
Pre-clerk Year 2	19 (59.4%)	20 (35.7%)
First year of Clerkship	4 (12.5%)	12 (21.4%)
Second year of clerkship	8 (25%)	19 (31.9%)

Mann-Whitney U Test of Likert Scale data showed that learning objectives were better met for the in-person conference (Median = 6) versus virtually (Median = 6), p <.01, and for large group sessions held in-person (Median = 7) versus virtually (Median = 6), p <.05. There was no statistically significant difference in learning objectives being met when comparing in-person workshops (Median = 7) to virtual workshops (Median = 6), p =.698. Results are summarized in Figure 1.

Figure 1.

Histograms showing relative frequencies of answers on Likert Scales for A) Didactic sessions, B) Overall conference, C) Small-group workshops, with values ranging from 1 (Strongly Disagree that learning objectives were met) to 7 (Strongly Agree that learning objectives were met).

Survey takers agreed that the virtual nature of the conference made it more geographically and financially accessible (M = 6.32, SD = 1.15), and less susceptible to scheduling conflicts (M = 6.04, SD = 1.14), and somewhat agreed that the virtual nature of the conference limited their interaction time with participants and presenters (M = 5.04, SD = 1.43). In terms of satisfaction with other components of the conference, participants were most satisfied with the value for cost of the conference (M = 6.55, SD = 0.82), generally satisfied with Zoom as a virtual platform (M = 5.59, SD = 1.27), and felt somewhat satisfied with the level of technical support that could be provided (M = 5.24, SD = 1.65). Likert scale data are summarized in Figures 2 and 3.

Figure 2.

Likert Scale data survey data ranging from 1 (Strongly Disagree) to 7 (Strongly Agree) for various components of the virtual conference.

Figure 3.

Likert Scale data survey data ranging from 1 (Highly Dissatisfied / Poor) to 7 (Highly Satisfied / Excellent) for various aspects of the virtual conference.

Discussion

While the effectiveness of virtual medical education has been demonstrated, there remains a paucity of information regarding the effectiveness of virtual scientific conferences.^16,17

The results of this study demonstrated that learning objectives were better met in the in-person conference both overall and in large group sessions. Interestingly however, there was no statistically significant difference between groups for small-group workshop settings. Though no studies have directly compared virtual and in-person conferences, these findings contrast with Wilcha (2020) and Pei et al. (2019) who found students to be generally satisfied with virtual teaching as an alternative to traditional delivery methods.^10,18 Findings appear to be at least partially explained by survey data which highlighted a decreased potential for interaction with presenters and attendees in a virtual setting, and is in keeping with similar concerns in previous studies.^{6, 7,19, 20} Interestingly, the lack of perceived interaction may also have downstream effects on other areas of engagement, such as decreased social media engagement in the recent American Cardiology Congress conference.²¹ This may explain why small-group sessions were comparable between the two conferences, as they subverted these issues of interaction and engagement, and may indicate an incentive for future virtual conferences to shift away from traditionally scheduling involving large-group lectures.

Despite these shortcomings, our results showed that virtual conferences improved overall accessibility by mitigating financial, geographic, and scheduling barriers and are in keeping with observations made from other virtual conferences.^{6,7, 19-22} Ease of accessibility is important not just for increased dissemination of information, but also to improve representation of groups traditionally underrepresented in in-person conferences such as individuals with young children.²³ In our study, the increase in inclusivity and diversity was demonstrated by a statistically significant increased variety in geographical attendance of the virtual Internal Medicine conference. Prior to transitioning to a virtual platform, attendance in past years has historically been dominated by attendees from the host institution.

Utilization of a virtual platform can increase program scheduling flexibility, but also introduce other challenges.²⁵ Our conference faced several client and end-user difficulties, most of which were unforeseen and occurred during the first day. Our sentiments were echoed in a recent review by O’Doherty et al (2018) who identified technical difficulties as a common limitation to online medical education.²⁵ Future planners aiming to implement a virtual platform are advised to extensively stress test the technical aspects of the conference in order to identify and later prevent potential technical issues.

Limitations

Although the majority of workshops offered were thematically identical between the two conferences, changes in content or presenter could have been a potential confounding variable impacting the results of the survey. The reliance on survey data additionally makes data collected at risk of nonresponse bias and sampling bias, which may be more pronounced with the low response rate. Finally, survey items relied solely on the subjective interpretation of items by responders, which can vary person to person depending on their personal learning objectives. This may be resolved in the future with the use of objective measures of educational impact, such as in the form of testing for memory retention following the event. This limitation, in addition to the fact that the survey used in this study lacks validity evidence, may additionally impact the overall validity and generalizability of findings.

Conclusions

Data from our study provide additional insights into the advantages and disadvantages of virtual conferences. While a vast array of benefits relevant to accessibility were found, we also found that the overall educational value of virtual conferences appeared to be less favourable than in-person conferences, and may be attributed to a perceived lack of potential for interaction and networking. Transitioning from a more traditional schedule to one that utilizes a greater presence of small group sessions may additionally rectify the overall conference experience, as these were found to offer comparative educational effectiveness. While we believe that the data provide valuable insight for educators considering virtual conferences as an educational platform, further research in the area is encouraged as there is much more to know about this area and virtual conferences will inevitably remain prevalent long after COVID-19.