Effects of voluntary attendance patterns on first-year medical students’ wellness and academic performance during COVID-19

Abstract

There is increasing evidence that attendance is not a reliable predictor of academic performance, which invites the question of whether attendance may have alternative positive effects such as increased student wellness. While conducting an observational cohort study on the relationship of attendance to performance and wellness, the COVID-19 pandemic lockdown changed the ability of students to attend class. Matriculating medical students were recruited to participate at the beginning of their first year in August 2019. In-person attendance was collected prospectively until the lockdown began in March 2020. This was then correlated with performance on course final examinations and responses to two separate incidences of a survey on stress, burnout, social isolation, and loneliness. The first survey was deployed January through February 2020 (pre-COVID) and the second June 2020 (during COVID lockdown). Attendance declined across the year (76.0%–25.0%). There was a small positive correlation of attendance to performance in module 1 (r = 0.235, P = 0.035) with none in the remaining two modules (P = 0.870, P = 0.410). The high attenders at the time of the lockdown had no decline in their performance when attendance was no longer possible. Attendance negatively correlated with stress and loneliness but not with the remainder of the wellness metrics. There was no significant change in any of the wellness metrics between the pre-COVID and during COVID time points. While the data support the conclusion that attendance is not required to maintain performance in this population, these data suggest a small protective effect against stress and loneliness.

INTRODUCTION

Medical education has undergone a remodeling during the last two decades. The rise of the internet has been accompanied by the ability to distribute and access aspects of the medical curriculum provided by the university they are attending via the internet. This change in mode of delivery of information has allowed easier access and flexibility and bestowed a certain degree of accountability on students (1). Students are no longer required to attend every class session in person, and they can access the same curriculum information from the comfort of their own home, a coffee shop, or even 100 miles away. While there is no question about the ease and flexibility electronically accessible curriculum material provides, its effectiveness in providing for academic success and the role it plays in students’ overall wellness is less clear.

A large meta-analysis demonstrated that in-person attendance is a strong predictor of academic success of undergraduate students based on studies that spanned over decades (2). In-class attendance was thought to be a marker of positive study habits, study skills, and attitudes (2, 3). These findings have been the basis of strong recommendations to students to attend classes if they want to succeed. The question now is does this apply to the current medical education environment?

A recent study was conducted by our research team at the University of Central Florida (UCF) College of Medicine to assess the effects of attendance at nonmandatory sessions in a single course within an undergraduate medical education setting. The sessions without mandatory attendance included a variety of pedagogies. While the majority of them were large group, primarily lecture-based sessions, a significant minority of them were small-group case-based learning activities. The results of this study suggested attendance at nonmandatory sessions was not correlated with academic success during a 6-wk course that provided online access to all the materials and recordings of nonmandatory sessions (4). In a similar cohort study conducted with second year medical students at the University of California (UC) Davis School of Medicine, Eisen et al. (5) showed a similar lack of association of attendance and performance on the final exam in a dermatology course. However, the studies conducted at UCF College of Medicine and UC Davis School of Medicine did not account for longevity of academic performance within the M.D. program or attendance habits over a sustained period. A scoping review conducted on 45 articles related to online curriculum use in undergraduate medical education concluded use of online curriculum materials was correlated with improved learning outcomes (6). While the evidence is mounting that attendance does not predict performance in medical education, the literature also includes studies that suggest the opposite. Laird-Fick et al. (7) looked at the attendance of medical students at flipped classroom style sessions during a 24-wk course, the results concluded that students who voluntarily attended most of the sessions performed significantly better on assessments than those who missed three or more of the sessions. Therefore, the current research on the effect of attendance on performance in medical school is mixed, with no single definitive relationship and results that may be context dependent.

If students are able to access all necessary resources to perform well on knowledge-based assessments outside of the traditional in-person classroom, what are the benefits of attendance at nonmandatory sessions? It has been suggested that the in-person learning environment is valuable above and beyond knowledge acquisition for many reasons such as the collaborative learning environment and professional identity development (8, 9). In addition, our faculty have expressed concerns about other downsides of the lack of attendance. Will these students be able to build the same sense of community? Will these students develop a sense of social isolation and loneliness when they are spending a majority of their time studying from online materials? These questions that faculty propose are based on anecdotal observation that this is happening. Students can, however, interact with their fellow students outside of the formal classroom, but the faculty do not necessarily see this happening, rather they see that students are not attending. The lingering question that follows these anecdotal reports and observations is whether they are part of a larger pattern of social isolation and loneliness or if they are just isolated incidences.

Another major concern when looking at student well-being is the development of burnout. Up to 35% of medical students in M.D. programs across the United States experience what is known as burnout syndrome (10). Perceived social support is a protective factor against stressors that contribute to burnout (10, 11). The faculty’s concerns about learning primarily from online materials and the possibility of this leading to social isolation, and therefore a lack of social support, are in part due to their concerns that this can increase the likelihood of developing burnout. Therefore, the follow-up question becomes: is in-person attendance associated with more or less burnout?

Given that many of the studies on attendance and performance in medical education were conducted on relatively short courses/modules and the mixed outcome of those studies, further investigation with a longer duration over the preclinical curriculum seemed warranted. In addition, the potential link between nonmandatory attendance, burnout, and social isolation has not been characterized in the literature. Therefore, the original goal of this study was to evaluate how academic performance and wellness were affected by voluntary attendance at nonmandatory sessions by prospectively collecting attendance across the entire 2 yr of the preclinical curriculum at the UCF College of Medicine M.D. program. However, 8 mo after the study began, the COVID-19 pandemic interrupted all in person activities across the country and at UCF. Beginning in the middle of March 2020 and spanning the remainder of the academic year, all students were required to adopt a new learning environment where all aspects of the M.D. program, mandatory and nonmandatory, were delivered from an online platform. The students in this study who had been attending nonmandatory in-person sessions were now only given the option of watching prerecorded sessions online. This presented a new opportunity for the study investigators to assess the effects of attendance at nonmandatory sessions on wellness and academic performance by looking at these factors before and after the onset of the pandemic. The present study aims to explore these potential relationships and acquire evidence on ways to better advise preclinical medical students, not only in their personal and professional development during normal academic conditions but also circumstances precipitated by a pandemic.

METHODS

Study Design

This is an observational cohort study using mixed methods. In-person classroom attendance was measured for all nonmandatory sessions before the COVID-19 lockdown. Attendance was related to two broad sets of outcome measures: 1) summative assessment scores, and 2) surveys measuring student wellness. The COVID-19 lockdown event forced all students to become online learners, depriving regular attenders of their usual pattern of classroom participation. This presented the opportunity to assess changes in performance and wellness before and after the COVID-19 lockdown. The main hypotheses tested were 1) classroom attendance rate positively predicts academic performance, 2) classroom attendance rate positively predicts student wellness and protects against burnout, and 3) loss of classroom attendance during COVID-19 would more adversely affect students who previously attended class.

Educational Context

The University of Central Florida, College of Medicine (CoM) M.D. degree is a 4-yr program. The first two preclerkship years are an integrated basic science curriculum that begins with normal body processes and transitions into a body systems-based pathophysiology approach during the second year. Students must pass the USMLE Step 1 examination at the end of year 2 before moving into the clerkships and clinical electives in years 3 and 4. The focus of this study was the first academic year, which is shown in more detail in Fig. 1. The first two modules (Human Body-1 and -2) covered biochemistry, genetics, physiology and anatomy, and were completed before the interruption caused by the COVID-19 pandemic. At the time of the COVID-19 lockdown order, students were studying the Human Body-3 module, covering immunology, microbiology and associated pharmacology. This module was interrupted approximately halfway through, after which the teaching was moved completely online. The final didactic module of the year was the first body system course, covering hematology and oncology, and was wholly delivered online. The aforementioned courses were selected for this study because all had a primary focus on the competency of medical knowledge, all consisted of similar pedagogy and assessment, and together they provided a continuum to study phenomena before, during and after the COVID-19 lockdown order. Figure 1 shows three other courses running during the first year: Practice of Medicine (a clinical skills course), Focused Individual Research Experience (“FIRE”), and Psychosocial Issues in Healthcare; these three courses were not included in this study because of their unique goals and different methods of pedagogy and assessment.

Figure 1.

Schematic of the first year of medical school (M1) curriculum showing the modules included in the study and the timing of the COVID-19 lockdown.

Pedagogy and Assessment

Regular classes consisted of a variety of pedagogical approaches including conventional lectures, web-based self-learning modules, team-based learning (TBL), case-based learning, and simulated clinical encounters. Sessions with nonmandatory attendance were primarily in large classrooms and lecture based. However, there were variable amounts of problem solving and flipped classroom sessions intermixed with the conventional lectures in each module depending on the individual faculty member’s preference. Standard practice in the M.D. program is that all nonmandatory lecture-based sessions are video recorded and then provided online. The recordings are provided via a learning management system and consist of an audio/video recording of the faculty presenter with a video capture of the computer-generated presentation. When classes were suddenly forced to go virtual, with only 1-wk notice, the faculty adopted online pedagogies that included students watching prerecorded lectures followed by online discussions. Live interactions occurred most days in small and large groups via the Zoom platform. Online sessions included clicker sessions for the whole class as well as use of breakout small group discussions using case-based learning. The standard performance outcome measure used for each course was the student score on a summative final examination. The 16-wk module 2 used a combined score from a midterm and final examination. Examinations consisted of 100–140 single-best answer multiple-choice examinations using clinical vignettes in a format like that used by the National Board of Medical Examiners. In the first two pre-COVID-19 classes, exams were computer-based but were deployed face-to-face via the program ExamSoft in a proctored classroom environment. For the final two classes, exams were provided to students online in their place of residence using the secure system Respondus. At UCF CoM, students normally receive module grades on an ABC-F scale, and this was the case for modules 1 and 2. However, the curriculum committee approved changing the grading in COVID-affected courses to a Pass/Fail grade; the passing standard in both systems was 70%.

Participants and Recruitment

Eligible participants were all 120 first-year medical students in the 2019–2020 academic year. In August 2019, during the new medical student’s orientation, the study investigators gave a brief presentation of the goals of the study and the requirements of participation. An online survey was shared with the students via email for them to consent to the de-identified use of their performance records and attendance logs, and to receive online wellness surveys during the study; students were informed they would receive an incentive payment of $10 per hour for survey participation. Eighty-two students gave informed consent to join the study.

Attendance and Performance Data Collection

A Bluetooth-enabled smartphone application, Spotter EDU, was used to automatically record in-class attendance. This automated system was able to accurately capture the first time a student entered the classroom on a given day but could not record a new attendance event for back-to-back lessons in the same room unless a student left then reentered the classroom. Therefore, our attendance data are expressed as the number of course days on which a student recorded attendance at a nonmandatory session. These data were accessed and maintained by the CoM Offices of Assessment, Planning, and Knowledge Management, as part of regular course monitoring activities. These offices also maintained the academic performance records and survey responses of students and provided researchers with de-identified files for data analysis.

Survey Assessment of Student Wellness and Burnout

Students were asked to complete two surveys during the 2019–2020 academic year, detailed in Table 1. The survey was a 69-item survey that was adapted from the following published and validated instruments: the Graduate Stress Inventory–Revised (GSI-R) (12), the Modified School Burnout Inventory (SBI) (13), the CDC’s Healthy Days Measure (14), the Pearlin Mastery Scale (15), the Multidimensional Scale of Perceived Social Support (MSPSS) (16), the De Jong Gierveld 6-Item Loneliness Scale (17), and the Lubben Social Network Scale–6 item (18). The first survey, a part of the original study design, was deployed pre-COVID from January 6th to February 28th, 2020. The second survey was added to the original study design and was deployed at the end of the first year of medical school (M1) from June 3rd 2020 to June 17th 2020 ∼3 mo after the COVID-19 lockdown began. Email reminders were sent weekly during the survey periods. The survey was administered using the Qualtrics survey system. All surveys were coded and the results calculated as described in the original publications, except for the GSI-R, which we inadvertently coded using a 1–8 Likert scale rather than the original 1–7 Likert scale.

Table 1.

Wellness survey items

Survey Items [Maximum Possible Score]	Pre-COVID Median (IQR) (n = 44)	During COVID Median (IQR) (n = 39)	P
Graduate Stress Inventory-Revised (GSI-R) [160]	58 (14)	60 (21)	0.564
Please rate the extent to which the following events have caused stress in your medical school education:	32 (7.5)	34 (12)	0.652
GSI-R Academic Stress Subscale [56]
1. Fulfilling responsibilities at home and at school
2. Taking exams
3. Fear of failing to meet program expectations
4. Handling relationships
5. Handling the academic workload
6. Writing papers
7. Meeting deadlines for course assignments
GSI-R Environmental Stress Subscale [64]	14 (8)	15 (7)	0.664
8. Trying to meet peers of your race/ethnicity on campus
9. Finding support groups sensitive to your needs
10. Participating in class
11. Meeting with faculty
12. Living in the local community
13. Peers treating you unlike the way they treat each other
14. Faculty treating differently from your peers
15. Adjusting to the medical school/hospital environment
GSI-R Family/Monetary Stress Subscale [40]	13 (8)	12 (9.5)	0.831
16. Being obligated to participate in family functions
17. Arranging childcare
18. Paying monthly expenses
19. Family having money problems
20. Being obligated (now or in the future) to repay loans
Modified School Burnout Inventory (SBI) [54]	25 (11)	24 (12)	0.392
1. I feel overwhelmed by the M.D. program
2. I feel a lack of motivation in the M.D. program and often think of giving up
3. I often have feelings of inadequacy in the M.D. program
4. I often sleep badly because of matters related to the M.D. program
5. I feel that I am losing interest in the M.D. program
6. I am continually wondering whether the M.D. program has any meaning
7. I brood over matters related to the M.D. program a lot during my free time
8. I used to have higher expectations of the M.D. program than I do now
9. The pressure of the M.D. program causes me problems in my close relationships with others
CDC Healthy Days Measure (CDC HRQOL–4)	2 (1)	2 (1)	0.643
1. Would you say that IN GENERAL your health is: a. Excellent, b. Very good c. Good d. Fair e. Poor [Best score =1]
2. Now thinking about your PHYSICAL HEALTH, which includes physical illness and injury, for how many days during the past 30 days was your physical health not good? _______	2 (5)	0 (2)	0.124
3. Now thinking about your MENTAL HEALTH, which includes stress, depression, and problems with emotions, for how many days during the past 30 days was your mental health not good? _______	5 (8)	4 (6)	0.249
4. During the past 30 days, for about how many days did poor physical or mental health keep you from doing your usual activities, such as self-care, work, or recreation? _______	7 (10)	5 (8)	0.124
Pearlin Mastery Scale [28]	24 (5)	24 (5)	0.738
For the following items: how strongly do you agree or disagree with these statements about yourself?
1. There is really no way I can solve some of the problems I have.
2. Sometimes I feel that I’m being pushed around in life.
3. I have little control over the things that happen to me.
4. I can do just about anything I really set my mind to.
5. I often feel helpless in dealing with the problems of life.
6. What happens to me in the future mostly depends on me.
7. There is little I can do to change many of the important things in my life.
Multidimensional Scale of Perceived Social Support (MSPSS) [84]	75 (11)	76 (13)	0.748
How strongly do you agree or disagree with these statements about yourself?
1. There is a special person who is around when I am in need.
2. There is a special person with whom I can share my joys and sorrows.
3. My family really tries to help me.
4. I get the emotional help and support I need from my family.
5. I have a special person who is a real source of comfort to me.
6. My friends really try to help me.
7. I can count on my friends when things go wrong.
8. I can talk about my problems with my family.
9. I have friends with whom I can share my joys and sorrows.
10. There is a special person in my life who cares about my feelings.
11. My family is willing to help me make decisions.
12. I can talk about my problems with my friends.
De Jong Gierveld 6-Item Loneliness Scale [6]	2 (3)	2 (3)	0.527
In answering the following questions, it is best to think of your life as it generally is now (we all have some good or bad days):
1. I experience a general sense of emptiness
2. There are plenty of people I can rely on when I have problems
3. There are many people I can trust completely
4. I miss having people around me
5. There are enough people I feel close to
6. I often feel rejected
Lubben Social Network Scale [30]	19 (3)	19 (5)	0.728
FAMILY: Considering the people to whom you are related, either by birth or marriage . . .
1. How many relatives do you see or hear from at least once a month?
2. How many relatives do you feel at ease with that you can talk about private matters?
3. How many relatives do you feel close to such that you could call on them for help?FRIENDSHIPS: Considering all of your friends, including those who live in your neighborhood . . .
4. How many of your friends do you see or hear from at least once a month?
5. How many friends do you feel at ease with that you can talk about private matters?
6. How many friends do you feel close to such that you could call on them for help?

IQR, interquartile range; CDC, Centers for Disease Control and Prevention. P values are displayed for Mann-Whitney U tests comparing survey results before and during the COVID-19 lockdown.

Statistical Analysis

Data analysis was conducted on data collected before and during COVID-19, which includes demographic data, attendance, summative exam performance, and the different aspects of wellness described above. Excel and IBM SPSS Statistics 25 were used to conduct the data analysis. Class demographic information included age as well as the percentages of students who were male or female and percentages who were either residents or nonresidents of Florida. Attendance data were expressed as the number of days when a student recorded attendance at a nonmandatory session as a percentage of the total possible days of attendance. Performance was expressed as summative examination score percentages. Wellness metrics were described using Likert scales. Data are reported as medians and interquartile ranges.

Shapiro-Wilk tests revealed that overall attendance and performance scores were not normally distributed. Therefore, the relationship between face-to-face attendance and performance in each Module was evaluated using Spearman’s correlation. To determine the impact of losing face-to-face attendance during module 3, the class was sorted into two groups: “High Attenders (HA)” with more than 50% attendance rate in the first half of the module and “Low Attenders (LA)” with less than a 50% attendance rate over this period. Nonparametric Kruskal-Wallis analysis of variance was applied to assess if the summative assessment scores for these two groups differed across modules 1–4; pairwise comparison of High Attenders versus Low Attenders was applied using Mann-Whitney U tests.

Internal consistencies for each section of the wellness survey were tested using the Cronbach’s alpha statistic. Forty-four students completed the pre-COVID survey, 39 students completed the survey during COVID-19, and 31 students completed both surveys. All three groups were analyzed separately and wellness score differences were calculated for each variable in the pre-post group. Simple descriptive analysis between the different wellness outcomes and demographics were explored initially. Pairwise comparisons for each corresponding subscale of the survey were made before and during the lockdown, by applying Mann-Whitney U tests. Regression analysis was conducted to predict participants’ wellness based on attendance while also controlling for possible confounders. The possible confounders were the available demographics: sex, in-state residence, and age. Normality assumptions of all continuous wellness variables were examined using Shapiro-Wilk test. The wellness variables that passed the normality test were analyzed using a multivariate regression. Otherwise, a Poisson linear regression was conducted if they followed other assumptions.

RESULTS

Sample Demographics

A total of 82 students out of 120 matriculated students consented to participate in the study. Eighty-one students had a complete examination record for inclusion in correlation studies. The demographics of study group were almost identical to the whole matriculating class, with an average age of 24.1 yr (SD = 2.6), with 51% female, 49% male, 71% in-state students, and 29% from out-of-state. Forty-four of 82 students completed the pre-COVID survey; their average age was 24.6 yr (SD = 3.1), with 44% female, 56% male, 64% in-state students, and 36% from out-of-state. Thirty-nine of 82 students completed the second survey during the COVID lockdown; their average age was 24.0 yr (SD = 1.6), with 51% female, 49% male, 66% in-state students, and 34% from out-of-state.

Relationships between Attendance and Performance

The average number of days of face-to-face classroom attendance declined across the year. Expressed as median (interquartile range): students attended 76.0 (36.0)% of days in module 1, 37.3 (43.3)% in module 2 and 25.0 (45.0)% of available days in the first half of module 3. No face-to-face attendance was possible in the post-COVID lockdown module 4. Nonparametric repeated measures ANOVA indicated the decline in median attendance with time across modules 1–3 was statistically significant (Friedman statistic = 83.9, P < 0.0001). Figure 2 summarizes the relationships between face-to-face attendance and performance, for the 81 students with complete examination records in modules 1–3. There was a weak but significant positive correlation between attendance and performance in the first class of the year (module 1: Spearman’s rho = 0.235, P = 0.035). However, there was no significant correlation in module 2 (Spearman’s rho = 0.018, P = 0.870) or module 3 (Spearman’s rho = 0.092, P = 0.410).

Figure 2.

Spearman’s correlation between face-to-face classroom attendance and examination Performance for modules 1–3. Module 1, r = 0.235, P = 0.035*; module 2, r = 0.018, P = 0.695; module 3, r = 0.092, P = 0.410; n = 81 complete student records in each case

Effect of Loss of Attendance Due to COVID Lockdown

Figure 3 shows box and whiskers plots for examination performances across the year for modules 1–4, for students designated as either High Attenders or Low Attenders at the time when the COVID lockdown occurred. Nineteen students met the criterion of >50% attendance during the first half of module 3 and as a group had a median (and interquartile range) attendance of 67.7 (21.0)% days attended. The other 62 students with <50% attendance had an attendance of 9.7 (16.1)% days attended. In general, students in both groups performed to a high level across all modules, with the median final exam score ranging from 87.3% to 92.9%. There was no significant difference in median examinations scores across modules 1–4 for either the LA group (Kruskal-Wallis test statistic = 1.13, P = 0.771) or the HA group (Kruskal-Wallis test statistic = 5.37, P = 0.147). Pairwise comparisons between LA and HA students within each module also showed no significant differences in median examination scores at the 5% level, although the difference between median scores was close to significant for module 4, when all students were forced to be only working online: module 1 (Mann-Whitney U statistic = 462.0, P = 0.158), module 2 (Mann-Whitney U statistic = 549.5, P = 0.664), module 3 (Mann-Whitney U statistic = 458.5, P = 0.147), and module 4 (Mann-Whitney U statistic = 417.5, P = 0.056).

Figure 3.

Box and whiskers plots comparing examination performance across the year for students with high or low attendance at the time of the COVID-19 Lockdown. HA, high attendance (>50% of nonmandatory lessons attended in the first part of module 3); LA, low attendance (<50% of nonmandatory lessons attended in the first part of module 3). No significant differences were found across modules for either HA or LA students or for pairwise comparisons of HA versus LA within any of the modules (Kruskal-Wallis ANOVA plus Mann-Whitney U tests, see text for P values).

Student Wellness and Burnout Survey Responses

Internal consistency was acceptable for each subpart of the survey shown in Table 1, and on each of the two occasions it was offered, with Cronbach alpha statistics ranging from a minimum of 0.64 to a maximum of 0.94. The sample of students who completed the surveys (both pre- and during COVID) were characterized as generally well, with a low level of burnout. For example, the scores data shown in Table 1 for the Graduate Stress Scales correspond to median stress levels between 2 and 5 on an 8-point scale where 1= not at all stressful to 8 = extremely stressful. Similarly, the SBI burnout scores correspond to a median response of “Partly Disagree” to statements related to feelings of exhaustion, cynicism and inadequacy. On the Centers for Disease Control Healthy Measures scale, students rated their general health as very good on average, noting an average of 0–2 out of 30 days with physical ill health. Of more concern, they expressed an average of 4–5 out of 30 days with poor mental health. Students strongly agreed with statements describing their positive feelings of mastery. In terms of social support, the MSPSS scores shown in Table 1 correspond to very strong agreement that students have friends and family providing them with support. Similarly, students scored an average of only 2 out of 6 on the Loneliness Scale and had an average of three to four close friends or relatives they interacted with on the social network scale. As shown in Table 1, the scores collected 1–2 mo before the COVID lockdown were very similar to scores collected ∼3 mo after the lockdown. No significant differences were found in any category when treating the surveys as independent samples and utilizing all responses (see P values in Table 1). For the 31 students who completed both surveys, paired analyses also confirmed no pre-post differences in any of the survey categories (data not shown).

Relationship between Attendance and Wellness

Various multiple regressions were calculated to predict participant wellness based upon their face-to-face attendance rate (see Table 2). When relating attendance rate to the wellness variables obtained pre-COVID, the results suggested some improved wellness associated with attendance. For example, a one-day increase in attendance was associated with a 0.331-point decrease in the Graduate Stress score (β = −0.331, P = 0.048, R² = 0.217) and a 0.339-day decrease in days lost to ill health (β = −0.339, P = 0.043, R² = 0.216). The data requiring Poisson regressions clarified that a 1-day increase in attendance was associated with a significant decrease in stress in the family subscale of the GSI (B = −0.01, P = < 0.001) as well as an improved loneliness score (B = −0.015, P = 0.008). However, no other scales showed a significant predictive association between attendance and wellness/burnout. When exploring the effect of attendance on wellness using the post-COVID lockdown survey data, it was again observed that that a day of attendance was associated with a decrease in the GSI overall stress score (β = −0.482, P = 0.029, R² = 0.203), although the rest of the regressions did not demonstrate any association between attendance and other aspects of wellness. Additional regressions were explored using the data for the 31 students who completed both surveys to assess if attendance before COVID affected wellness after the COVID lockdown. These data showed that a 1-day increase in attendance before COVID was associated with small 0.007-point decrease in the days with mental health problems after COVID lockdown (B = −0.007, P = 0.004), although no other significant associations were found (data not shown).

Table 2.

Regression analysis of the relationship between wellness survey variables and voluntary attendance before and after the COVID-19 lockdown

Dependent Variable Name	Pre-COVID				During COVID Lockdown
	Regression coefficients			P value	Regression coefficients			P value
	B	SE (B)	β		B	SE (B)	β
Overall GSI-R Score	−0.301	0.147	−0.331	0.048*	0.088	0.160	0.123	0.586
Academic Stress Subscale	−0.082	0.086	−0.165	0.347	−0.018	0.112	−0.036	0.876
Environmental Stress Subscale	−0.105	0.053	−0.342	0.054	0.011	0.065	0.038	0.870
Family/Monetary Stress Subscale	−0.01	0.12	N/A	0.000*‡	0.095	0.050	0.405	0.067
Overall SBI Score	−0.065	0.059	−0.195	0.273	−0.073	0.066	−0.241	0.281
Exhaustion Subscale	−0.060	0.033	−0.305	0.079	−0.011	0.042	−0.062	0.786
Cynicism Subscale	0.000	0.003	N/A	0.997‡	−0.046	0.020	−0.482	0.029*
Inadequacy Subscale	−0.006	0.018	−0.055	0.761	−0.016	0.023	−0.144	0.502
CDC: Days of Poor Physical Health	−0.035	0.027	−0.200	0.208	−0.014	0.058	−0.049	0.810
CDC: Days of Poor Mental Health	−0.079	0.056	−0.238	0.168	−0.017	0.066	−0.059	0.793
CDC: Lost Days Due to Poor Health	−0.081	0.039	−0.339	0.043*	0.011	0.049	0.048	0.827
Pearlin Mastery Scale Score	0.000	0.002	N/A	0.942‡	0.035	0.033	0.228	0.312
Overall MSPSS Score	0.001	0.001	N/A	0.189‡	−0.014	0.084	−0.037	0.874
Social Support Subscale	0.002	0.002	N/A	0.128‡	−0.033	0.053	−0.144	0.533
Family Support Subscale	0.001	0.002	N/A	0.501‡	−0.009	0.025	−0.082	0.724
Friend Support Subscale	0.003	0.031	0.019	0.916	0.029	0.029	0.204	0.339
Loneliness Score	−0.015	0.006	N/A	0.008*‡	0.005	0.015	0.067	0.764
Social Network Score	0.058	0.029	0.346	0.052	−0.029	0.037	−0.168	0.437

GSI-R, Graduate Stress Inventory–Revised; CDC, Centers for Disease Control and Prevention; MSPSS, Multidimensional Scale of Perceived Social Support; SBI, School Burnout Inventory; B: beta, unstandardized regression coefficient; SE (B): SE of unstandardized beta; β: standardized beta. *Statistical significance. ‡Poisson regression due to nonnormality of data.

DISCUSSION

In general, the data support the conclusion that attendance does not reliably predict performance in the preclinical curriculum. The students performed well on the knowledge-based assessments regardless of their in-person attendance level across the academic year. The natural experiment that occurred with the COVID lockdown allowed us to look at what happens to the students who choose to attend but are no longer able to. The high attendance students were now only able to interact with the nonmandatory portion of the formal curriculum through the prerecorded lectures, the same as the low attendance ones. If it was the attendance itself that was the main reason for their good performance, you would expect that these students’ performance would drop when attendance was taken away. This did not occur but instead they maintained a high level of performance across the academic year (Fig. 3).

Comparison of performance between the different modules is challenging since they are independent courses with different groups of faculty setting standards. In addition, adjustments were made to the assessments due to the external stressors imposed by the COVID-19 lockdown, in particular the change in the grading system to pass/fail for modules 3 and 4. We adjusted for these confounders by comparing the high attenders to the low attenders at the time of the lockdown within each course. There was no significant difference between these two groups across the academic year. In the module that was split in half by the lockdown and in the one that was completely online, the high attendance students maintained their performance when compared with the low attendance students. In fact, when they moved to online, there is a suggestion that the previously high attendance students’ performance was higher than the previously low attendance students (Fig. 3), again suggesting that classroom attendance per se is not a major determinant of performance on multiple choice question tests.

One interesting finding was that the first course of the year (module 1) did have a small yet statistically significant positive correlation of attendance to performance (R² = 0.055, P = 0.035). This course had a significantly higher level of attendance (76.0% of days) as compared with the courses later in the year (37.3% in module 2 and 25.0% in module 3). For many students, this would be their first graduate level course and the general sense at the beginning of the year is one of excitement to be in the M.D. program. This perceived sense of excitement is born out in the higher level of attendance. There is a strong “Big Sib” program at the CoM where the newly matriculated students are paired with one of the raising second-year students, which allows for a lot of advice on study habits to be passed down from these upperclassmen. The students can learn from these near peers that they do not need to attend to maintain their level of performance. As some the high-performing students choose to leave the classroom behind, this small correlation of attendance to performance is lost (as seen by the declining attendance without declining performance across the year). A previous study that looked at the students mid-way through their second-year in the M.D. program showed that students who do not attend nonmandatory sessions had higher levels of self-efficacy and the ability to self-regulate effort (4). They seem to have used this characteristic to find equally effective study habits that do not include attendance. We postulate that some of the students who are not attending at the beginning of the academic year are doing so not because they have a strong study strategy that does not include in-person attendance but because of a lower level of engagement with the material.

The literature shows a similar trend; studies that demonstrate a high level of attendance are more likely to show a correlation of attendance and performance than those with a lower level of attendance. Bamuhair et al. (19) showed a mean attendance of 86.1% with a strong correlation of attendance to performance (0.52; P < 0.001) in a PBL-based cardiology course in Saudi Arabia. This course required a minimum of 75% attendance to sit for the final exam. In an OB/GYN clinical rotation in Dublin, tutorial attendance was 90% and showed a strong correlation of tutorial attendance to performance (r = 0.57; P < 0.001) (20). Our previous study had a much lower overall attendance in a second-year Renal and Gastrointestinal systems module (24%) with no correlation of attendance to performance (4). Eisen et al (5) had a similarly low level of attendance (38.7%) in a dermatology course with no correlation of attendance to performance. While this collection of studies does not enlighten us as to why this is seen, it is interesting and suggests the need to look at what study strategies the students who are not attending have in both of these situations.

If attendance is not required for success on knowledge-based assessments, what is the driving force that leads the students who maintain a high level of attendance? We looked at one potential aspect of this by investigating the wellness metrics of social isolation, loneliness, and burnout in this population. A review of the literature indicates that the prevalence of burnout in preclinical students ranges from 29 to 75% (21). The subset of the students who completed our survey were overall a “well” and resilient group (Table 1). They had relatively low scores on the graduate stress, burnout, and loneliness scales and relatively high scores on mastery, social network, and social support scales with the paired statistics for the before and during COVID lockdown time points showing no significant change. These students were faced with the large stressor of the global pandemic on top of the stressful situation of being medical students (22). Despite this, they showed no change in their loneliness, social isolation, or burnout. This generation of students are digital natives and are used to connecting to each other in a virtual format (e.g., texting and social media), thus potentially allowing them to stay connected to each other during the pandemic in a way that they were already used to. Also, many of our students chose to return to reside with their family at this point in the pandemic potentially mitigating some of these factors. Overall, this is a generally socially connected group of students without significant burnout who were able to handle the changes imposed by the pandemic without significant deterioration in these areas. One limitation of this portion of the study is one of selection bias. It is likely that students who were struggling before or during the lockdown would not have had the motivation to complete the survey, thus preventing us from fully evaluating potential benefits of attendance related to quality of life.

Previous work has pointed to protective factors against burnout including engagement and problem solving (21), which one would expect to be enhanced with more classroom interactions. When looking to see how these wellness markers correlated with attendance, we see that most of them did not; however, there were a few small, but statistically significant, correlations (Table 2). Students with a higher level of in person attendance had lower scores on the Graduate Stress Index and the Loneliness scale. Although it did not quite reach significance, there was also a small positive correlation of attendance with the Social Network Scale (P = 0.052). This suggests that students are possibly getting some positive benefit from the social interaction with their peers in the classroom setting giving them a small protective benefit from graduate stress and loneliness. A frequently stated concern from faculty is that the students who are not attending are spending too much time alone in front of their computers and becoming socially isolated. Our current data do not tell us what the students who are not attending are doing with their time. They could be having plenty of interaction with their fellow students in study groups and other extracurricular activities. However, the faculty’s concerns are based on anecdotal reports from individuals that this type of isolation is occurring. Several studies have shown that social isolation is a risk factor for burnout in medical students (10, 11). While there are unaccounted for cofounding factors that prevent us from making a definitive conclusion, our data support the faculty’s concerns and suggests a potentially small protective benefit of classroom attendance.

The finding of decreased scores on the Graduate Stress index with attendance was also found in the second instance of the survey. This was deployed after the COVID lockdown began and after the academic year ended and therefore the students were no longer in the classroom environment. Therefore, the correlation was not with current attendance level but with past attendance level. The lack of a change in these scores when they were no longer in the classroom potentially suggests that this finding is not related to their level of attendance but to some other aspect of these student’s lives instead. Further investigation on the behavior of the nonattending students outside of the classroom would be required to make definitive conclusions for these findings.

As stated previously, this group of students is resilient as there was no change in the measured wellness metrics when using a paired analysis in the students who completed both surveys or when using an unpaired analysis and incorporating all survey respondents. Other investigators were in a similar position in that they were obtaining data on burnout before the pandemic and were able to compare it to second instance of their survey after the lockdown began. Balatov et al. (23) surveyed first through fifth-year medical students in Kazakhstan in October of 2019 and then repeated their survey in April of 2020. Their study demonstrated a decrease in the percentage of students who demonstrated burnout after the transition to online learning as compared with during the traditional curriculum (16.7% vs. 27.6%; P < 0.001) (23). Zis et al. (24) was conducting a cross-sectional study on burnout in all 6 yr of their program at the Medical School of the University of Cyprus. They surveyed their participants in January of 2020 and were able to repeat the survey in this study group in March. When looking at the total study group, there was no change in the percentage of students with burnout during COVID as compared with before (18.2% vs. 18.1%; P = 0.5) (24). In both circumstances, the data appeared to have been collected in such a way that did not allow for paired analysis, as opposed to our current study. In contrast, Zhang et al. (25) found high levels of learning burnout in medical students in Wuhan China (46.12%, n = 684 students). While they stated that this was higher than previously published levels of burnout, they did not have any internal pre-COVID data to compare it to Ref. 25. All of these groups were surveyed at the beginning of the pandemic when many places in the world were locking down to prevent a severe outcome and not necessary because of one currently happening. Wuhan China was where the pandemic began and was in a much worse state at this point of the pandemic than the other locations studied (26). This could explain the different outcome seen in China.

There are some limitations to this study. This is a single institution study which potentially limits its generalizability to other programs and settings. There was also a lack of granularity in the attendance data since the tracking system was only able to reliably capture the first session that they attended in any given day which limited the scope of these data. The occurrence of the pandemic is both a limitation and a benefit. The original study was not designed to investigate how these sudden and enforced changes in student behavior would affect wellness and performance. It was designed to see how the students’ personal choice to attend or not related to these outcomes over the course of the entire preclinical curriculum. However, by removing this choice from the student who did attend, the pandemic allowed us to strengthen our argument that it is not the level of attendance itself that is determining the student’s level of success on knowledge-based assessments. What these findings do suggest is that we need to further investigate how in-person attendance effects competencies outside of knowledge. For example, how is attendance related to communication skills or success on performance-based assessments? This could then better inform educators how to design their face-to-face sessions to capitalize on strengthening these areas.

GRANTS

Funding supplied by the University of Central Florida College of Medicine.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

J.S., A.E., J.D.K., and C.A.K. conceived and designed research; J.S., M.W., A.E., J.D.K., and C.A.K. analyzed data; J.S., M.W., A.E., J.D.K., and C.A.K. interpreted results of experiments; A.E. and J.D.K. prepared figures; J.S., M.W., A.E., J.D.K., and C.A.K. drafted manuscript; J.S., M.W., A.E., J.D.K., and C.A.K. edited and revised manuscript; J.S., M.W., A.E., J.D.K., and C.A.K. approved final version of manuscript.