Adapting a Medical School Cancer Research Education Program to the Virtual Environment: a Mixed-Methods Study

Abstract

With cancer incidence increasing worldwide, physicians with cancer research training are needed. The Scholars in Oncology-Associated Research (SOAR) cancer research education program was developed to train medical students in cancer research while exposing them to the breadth of clinical oncology. Due to the COVID-19 pandemic, SOAR transitioned from in-person in 2019 to virtual in 2020 and hybrid in 2021. This study investigates positive and negative aspects of the varying educational formats. A mixed-methods approach was used to evaluate the educational formats. Pre- and post-surveys were collected from participants to assess their understanding of cancer as a clinical and research discipline. Structured interviews were conducted across all three cohorts, and thematic analysis was used to generate themes. A total of 37 students participated in SOAR and completed surveys (2019 n = 11, 2020 n = 14, and 2021 n = 12), and 18 interviews were conducted. Understanding of oncology as a clinical (p < 0.01 for all) and research discipline (p < 0.01 for all) improved within all three cohorts. There was no difference between each cohort’s improvement in research understanding (p = 0.6). There was no difference between each cohort’s understanding of oncology-related disciplines as both clinical and research disciplines (p > 0.1 for all). Thematic analysis demonstrated that hybrid and in-person formats were favored over a completely virtual one. Our findings demonstrate that a medical student cancer research education program is effective using in-person or hybrid formats for research education, although virtual experiences may be suboptimal to learning about clinical oncology.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13187-023-02291-y.

Introduction

As the number of patients with a current or prior cancer diagnosis continues to grow worldwide, there is an increasing need for physicians with cancer research training to develop novel approaches to screening, diagnosis, therapy, hospice/palliative care, and survivorship. Moreover, the incidence of different cancers is projected to change in the next 20 years, which requires medical schools to engage students in cancer research while also exposing them to the myriad medical specialties that are involved with clinical cancer care [1]. Unfortunately, many medical schools fail to offer a comprehensive oncology educational experience [2].

To remedy this gap in programming, a single US medical school developed the SOAR cancer research education program in 2015 with the goal of training medical students in high-quality cancer research methodology while simultaneously exposing them to different avenues to pursue clinical cancer care and cancer research. Analysis of the 2015–2016 pilot program demonstrated that participation in SOAR improved participant understanding of oncology and its related specialties from both a clinical and research perspective [3]. Beginning in 2019, SOAR became a National Cancer Institute (NCI) cancer research education program. In the NCI-funded SOAR program, first-year medical students are engaged in an individually mentored 11-week cancer research experience that culminates in an oral presentation and internal manuscript submission. The cancer research experience is supplemented by in-depth exposure to aspects of clinical cancer care consisting of a survey seminar series on oncology and associated clinical specialties in relation to research opportunities, tumor board shadowing, interprofessional shadowing, and a journal club covering seminal research in oncology.

In 2020, due to the COVID-19 pandemic, this educational structure was modified to comply with social distancing requirements. While a difficult transition, this allowed for innovation of cancer research education programming with exploration of new educational formats to impart the SOAR experience. While there is some literature that has broadly examined this transition in other areas of medical education and beyond, there are few studies that compare longitudinal data from before the pandemic with in-person education and can compare a completely virtual versus a hybrid format [4, 5]. Additionally, there are no studies that examine this transition in the context of medical student cancer research education. Using a mixed methods approach, this study aims to fill this gap and understand how pandemic-related educational format changes impacted participant perspectives on a cancer research education program and whether the program remained as effective in the virtual and hybrid formats as the pre-pandemic in-person structure.

Methods

Curriculum and Adapting to the Pandemic

The SOAR program was developed using the backbone of a preexisting summer research experience between the first and second years of medical school. The curriculum was designed to train and engage medical students in cancer research while also exposing them to the interdisciplinary and interprofessional nature of clinical oncology and its evolving landscape. SOAR aims to provide an in-depth cancer research education experience supplemented by educational activities to instill a framework for multidisciplinary and interprofessional patient-centered oncologic care and research. To accomplish this, SOAR consists of multiple components: an 11-week mentored cancer research experience between the first and second years of medical school, a structured 10-week oncology seminar series, multidisciplinary tumor board attendance, interprofessional shadowing experiences, and a cancer research journal club (Fig. 1).

Fig. 1.

The Scholars in Oncology-Associated Research (SOAR) cancer research program annual cycle (IPE, interprofessional education; SRP, Pritzker School of Medicine Summer Research Program)

In 2019, the SOAR program was fully in-person. In 2020, because of the COVID-19 pandemic, all SOAR experiences were virtual, including research, tumor board experiences, and the seminar series. The interprofessional clinical shadowing programs were suspended. Due to the 100% virtual environment for 2020, funding was not provided by the NCI for the program, and all student cancer research experiences were funded by the University of Chicago Pritzker School of Medicine. In 2021, the cancer research experience was hybrid; the interprofessional shadowing returned to in-person, and the seminar series, tumor boards, and journal club remained virtual.

Assessments and Interviews

This study was determined to be exempt by the University of Chicago Institutional Review Board. Students participating in SOAR completed anonymous online self-assessments before and after the SOAR program. Pre-assessments collected demographic information of each participant. Both the pre- and post-assessments asked participants to rate their understanding of oncology and its related disciplines from both clinical and research perspectives. These questions were based on a 5-point Likert-type scale (1 = not at all, 2 = somewhat, 3 = moderately, 4 = quite, and 5 = extremely). Development of these questions was done with the help of external reviewers who are experienced in survey methodology. All responses were collected using REDCap (Research Electronic Data Capture) [6, 7].

In addition to the surveys, qualitative interviews were conducted with SOAR participants across all the cohorts. Interview participants were randomly selected from each cohort. Interviews were stopped at the point of data saturation when there was sufficient quantity and quality of data, and any additional data collection would not contribute new ideas [8].

Interviews were conducted from January 2022 to May 2022 and were between 15 and 20 min each. An interview protocol was developed with questions specific to each cohort’s experience (Online Resource 1). Prior to starting the interview, participants were given a brief introduction and an overview of the study’s purpose. The questions focused on gathering the participants’ most memorable moments of the SOAR program, information on how SOAR has professionally impacted them, and how effective an in-person curriculum is compared to a virtual or hybrid one. Interviews were conducted virtually, recorded, de-identified, and transcribed verbatim.

Data Analysis

A mixed-methods approach was used to analyze how participant understanding of cancer research and clinical care shifted and how curriculum changes due to the pandemic impacted this understanding.

The pre- and post-SOAR survey data was analyzed using quantitative methods with two primary objectives: to learn how an understanding of oncology and its related clinical and research disciplines changed within each year and how this understanding changed between each year because of pandemic-driven curriculum changes. Likert-type responses were analyzed with the Wilcoxon rank-sum test to examine how participant understanding of oncology and related specialties changed within each year’s program. The difference between the pre- and post-assessment responses for each question was compared using the Kruskal–Wallis test to examine change in understanding of oncology and its related specialties between the cohorts. The cutoff p-value for significance was calculated using a Bonferroni adjustment to correct for multiple tests.

Thematic analysis was used to analyze interview transcripts to curate themes related to SOAR’s effectiveness as a cancer research program and to assess cohort differences because of in-person, virtual, and hybrid curricula. Thematic analysis was deemed most appropriate because it offered the most flexibility in gathering different perspectives from participants. The coding method used has been described by Nowell et al. and is comprised of 6 different phases [9]. Phase 1 included getting a basic understanding of the data. Phase 2 involved the initial coding of repeated ideas in the data using ATLAS.ti software (Scientific Software Development GmbH, Berlin, Germany); sentences and phrases were used as units of analysis. Phase 3 involved combining the codes into larger, relevant themes across individual and multiple transcripts. Phase 4 included discussing the collected themes and consolidating them by the research team (O.R.V, H.A., and D.W.G.); this phase also involved understanding thematic outliers. In phase 5, themes were formally named. Lastly, in phase 6, researchers investigated the relationship between finalized themes, allowing for the creation of hypotheses. Medical student and cohort are denoted in the qualitative data using the following annotation: MS#-Format, where # represents the interview number within the cohort and format is replaced with the participant’s “curriculum format” (e.g., MS1-In-Person denotes medical student interview #1 who participated in SOAR when it was completely in-person). Representative quotes are edited for grammar and clarity.

Results

Pre- and Post-SOAR Assessments

A total of 37 students participated in SOAR from 2019 to 2021 (11 in 2019, 14 in 2020, and 12 in 2021; Table 1). In total, 100% of participants completed the pre- and post-SOAR surveys. Participant demographics, including age, undergraduate major, and gender, are reported in Table 1. Notably, 25 (68%) participants reported having the intention to pursue an oncologic specialty, 16 (43%) participants had previous oncology-related clinical experience, and 20 (54%) participants had previous oncology-related research experience.

Table 1.

SOAR participant demographics across all three cohorts

Participant demographics	Participants
	N = 37
Cohort, n (%)
2019	11 (29.7%)
2020	14 (37.8%)
2021	12 (32.4%)
Gender, n (%)
Male	18 (48.6%)
Female	19 (51.4%)
Age, n (%)
< 23	6 (16.2%)
23–25	27 (73%)
26–28	4 (10.8%)
Previous undergraduate major, n (%)
Biological sciences	25 (67.6%)
Physical sciences	3 (8.1%)
Social sciences	5 (13.5%)
Other	4 (10.8%)
Intend to pursue oncologic specialty, n (%)	25 (67.6%)
Previous oncology-related clinical experience, n (%)	16 (43.2%)
Previous oncology-related research experience, n (%)	20 (54.1%)

Self-reported understanding of oncology and many of its related specialties as clinical and research disciplines improved within all three cohorts, recapitulating previous findings from the analysis of the SOAR pilot program (Online Resource 2) [3]. There were various exceptions across the cohorts, but when comparing the magnitude of the difference in self-reported improvement in understanding between the three cohorts and each category, there was no difference in improvement seen between them (Table 2).

Table 2.

Mean difference in pre- and post-SOAR assessment scores for each category across all three cohorts. P value is a result of the Kruskal–Wallis test. Cutoff for determining significance set at 0.003 using Bonferroni adjustment to correct for multiple tests

Category	2019	2020	2021	P value
Oncology as a general clinical discipline	2.1	1.4	1.4	0.08
Oncology as a general research discipline	1.9	1.5	1.3	0.97
Cancer genetics as a general clinical discipline	1.4	1.1	1.3	0.90
Cancer genetics as a general research discipline	1.6	1.1	1.1	0.22
Medical oncology as a general clinical discipline	1.7	1.9	1.3	0.23
Medical oncology as a general research discipline	2.0	1.9	1.3	0.17
Pediatric oncology as a general clinical discipline	0.9	1.1	1.3	0.66
Pediatric oncology as a general research discipline	0.9	1.4	1.3	0.26
Radiation oncology as a general clinical discipline	1.5	1.6	1.2	0.70
Radiation oncology as a general research discipline	1.3	1.5	1.2	0.75
Surgical oncology as a general clinical discipline	1.4	1.3	0.8	0.52
Surgical oncology as a general research discipline	1.4	1.3	1.4	0.95
Hospice/palliative care as a general clinical discipline	1.6	0.9	1.2	0.45
Hospice/palliative care as a general research discipline	1.6	1.4	1.7	0.99
Survivorship as a general clinical discipline	2.5	2	1.9	0.57
Survivorship as a general research discipline	1.8	1.9	1.8	0.74
Drug development as a general research discipline	1.1	0.8	0.9	0.69
Global oncology as a general research discipline	2.0	1.5	1.7	0.50

Qualitative analysis revealed more nuanced differences between the SOAR cohorts based on educational format. 21 students were interviewed before reaching data saturation (6 from 2019/in-person, 8 from 2020/virtual, and 7 from 2021/hybrid). Below, each theme derived from thematic analysis is detailed and supplemented with the voices of participants. All themes and subthemes are summarized in Table 3.

Table 3.

Most common themes about SOAR and curriculum format across all interviews

Most common general themes about SOAR
1. Multidisciplinary nature of oncology
2. SOAR spring seminar series
3. Career exploration and understanding of oncology-related specialties
Most common themes about curriculum format
Themes	Subthemes
1. Benefits of in-person programming	Fostering informal connections Community
2. Benefits of virtual/hybrid programming	Flexibility and increased productivity as a researcher Improved mental health and well-being
3. Drawbacks of completely virtual programming	Difficult to remain engaged Loss of critical clinical exposure

Most Common General Themes About SOAR

Theme 1—The Multidisciplinary Nature of Oncology

Participants were asked about what they remembered most about SOAR. Many noted that they were able to better understand the multiple facets of oncology and how different fields intersect with cancer care.

“I most remember being surprised to learn about the different fields of oncology and the intersections or interplay between the field of oncology and other health realms. So, learning about all that oncology entails was surprising.”—MS2-Virtual

“I don’t think I knew how much went into the interdisciplinary aspect of [oncology] and how many different types of doctors are involved in care, what a team looks like, and how people come together. I learned a lot about that through tumor boards and shadowing opportunities that we had.”—MS12-Hybrid

One participant also noted the more humanistic side of delivering cancer care.

“I saw a lot more of the softer science side of oncology and how, in addition to the therapeutics, you provide a lot of social support and psychological support.”—MS11-In-Person

Theme 2—The SOAR Spring Seminar Series

Many participants also remembered the SOAR spring seminar series.

“What I remember the most is the Spring quarter course where we had the lectures. I probably remember that most because it was very clearly oncology and a separate elective that we were doing from the rest.”—MS12-In-Person

The seminar series allowed for an introduction to a variety of cancer-related specialties and research and opportunities to connect with potential clinical and research mentors.

“What I remember most is the lecture series. That is the component of the program that took the greatest amount of time in terms of scheduling outside of the research. Also, because I was really engaged in terms of meeting new people and scholars in the field.”—MS5-Hybrid

Theme 3—Career Exploration

Many participants also noted how imperative SOAR was to the development of their career and research goals.

“SOAR helped develop my understanding of what kinds of opportunities are available in the research realm of cancer. I didn’t have as much exposure to these kinds of research. The opportunities to see other’s research and having the seminar series and having other physicians talk about their research really allowed me to see what kinds of research I can do in my career.”—MS12-Hybrid

“[SOAR] made me aware that I can participate in oncology research through numerous avenues. In the past I thought that if I did not do hematology/oncology as a specialty I wouldn’t necessarily be able to participate in cancer research. For me personally, [SOAR] directed me toward gynecological oncology research.”—MS5-Hybrid

SOAR not only impacted participants’ scholarly goals, but it also allowed participants to gain a better grasp of the clinical side of oncology.

“SOAR was really the first time I was introduced into breast imaging specifically and how I could see myself pursuing that career…so this was helpful from a career development perspective.”—MS8-In-Person

This participant noted how important it was to hear from successful faculty to who they can relate to.

“SOAR opened up a horizon for me. I never saw myself as a surgeon or an oncologist broadly. I always thought I was going to do obstetrics/gynecology or primary care, so maybe Dr. X jumps out at me because I can imagine myself intersecting with oncology as a surgeon because she’s a woman and identifies as a woman. I think if the goal of SOAR is to expose folks in the early stages of their career to oncology broadly, I think it has achieved that goal.”—MS6-Virtual

Most Common Themes about Curriculum Format

Participants were then asked about their respective curriculum formats and how they would feel about in-person, virtual, and hybrid programming.

Theme 1—The Benefits of In-Person Programming

The primary benefits of in-person programming noted by participants included having the ability to engender informal connections with faculty and to create a sense of community with peers.

“I would say the main challenge was that you didn’t have as much of a community as you would have felt if it was in-person. SOAR wasn’t an isolating time, but I feel like it would have definitely been made better by being able to come in person at least once a week just to have more organic conversations with how everything is going and feeling more supportive.”—MS5-Virtual

“I think being in-person with folks offers a different type of connection. It offers the lingering after lecture and talking to someone with who you connected with about what they are doing or if you wanted to open a new opportunity.”—MS1-Hybrid

The following participant also noted a different level of engagement with the programming.

“I think that the great part of SOAR was engaging with the materials that lecturers brought in that made the lectures come to life”—MS9-In-Person

Theme 2—The Benefits of Virtual/Hybrid Programming

The primary subthemes noted about the benefits of virtual/hybrid programming included added flexibility and productivity in research.

“I feel like there’s a limited amount of time in the day, and I really did appreciate the flexibility [of virtual components] because I was also doing other things in the summer that were really time-consuming.”—MS12-Hybrid

Students in the virtual cohort noted the additional flexibility of the virtual environment.

“I think in the virtual format you have more flexibility in terms of doing research and balancing your other responsibilities. I think that’s the main benefit.”—MS2-Virtual

“Having the freedom to work from wherever is huge. It was really important to my mental health. I love this community and the patient population, and I’m so grateful to be doing my medical education here and I wouldn’t change it for the world; however, I miss my family and my friends and having to be here during the break would have been hard.”—MS3-Virtual

Students who participated in the hybrid format noted the balance of in-person and virtual potentially improved mental health and well-being after having spent the prior year in a 100% virtual learning environment.

“The flexibility probably was the best thing. I think that having that and getting to get out of Chicago, I know people that did but I didn’t. Having that option and having the fluidity in the scheduling made it very manageable and also customizable to my personal needs.”—MS6-Hybrid

Theme 3—The Drawbacks of Completely Virtual Programming

Participants also noted potential downsides to a completely virtual program, citing loss of engagement, and critical clinical exposure as primary drawbacks.

“Virtual seminars are fine, but then the in-person clinical relevance is important…things like shadowing and talking to physicians are things we missed out on.”—MS2-Virtual

“There tends to be more engagement when something is in person. It’s easier for people to tune out when it’s virtual so I generally get more from the experience if it’s in-person.”—MS8-Hybrid

“Just general Zoom exhaustion. It’s just hard to stay focused when all of our lectures are on Zoom, especially after first year [of medical school] when it was all hours and hours of lecture on Zoom each day, and then having to stay engaged for another lecture on Zoom.”—MS6-Virtual

Despite these drawbacks, some participants said a hybrid curriculum, with some tweaks, offers an ideal balance between flexibility, productivity, and in-person interaction.

“I don’t think everything virtual would have been good. I think this hybrid format was the sweet spot for me. I think everything virtual, like meeting with my mentor, is possible. My project makes that possible, but I wouldn’t have wanted to do that because I do get something out of coming in and talking to my mentor in person. And coming in and seeing the cluster group faculty and my peers in person. So having that option would have been very helpful. And also, things like interprofessional shadowing would not have been possible, which would take away from some of the mission of the program…”—MS12-Hybrid

“Flexibility was a great thing. If everything were required in person, it would be very regimented. I think that maybe I would not have been able to do some of the other things in the summer, so having a sort of hybrid model would be good in terms of keeping that in place.”—MS12-Virtual

Discussion

The primary objective of this study was to examine how the curriculum modifications to SOAR in the wake of the COVID-19 pandemic impacted participants and whether these lessons can guide future cancer research education programs. The survey and interview data demonstrate that all cohorts (fully in-person, fully virtual, and hybrid) subjectively improved in their understanding of oncology and many of its related specialties as both clinical and research disciplines. However, there was a suggestion of a discrepancy in the improvement of clinical understanding of oncology between the three cohorts made apparent while conducting interviews. The virtual (2020) cohort noted repeatedly how they were unable to translate their newfound knowledge about cancer research and clinical care to the clinical environment. Additionally, the interviews revealed the pitfalls of a completely virtual environment, including increased fatigue, loss of community, and inability to foster informal interactions. These drawbacks have been reported before but have not been thoroughly studied in the context of undergraduate medical education and cancer research education [5, 10]. Fortunately, there continues to be innovation in how medical educators are becoming more adept at delivering content through multiple formats [11, 12]. Additionally, researchers are reacting and adapting to the ongoing changes brought on by the pandemic [13, 14]. As innovation in research methodology continues to occur, there is more opportunity for flexibility in how programs such as SOAR can be implemented, especially if participants are engaging in clinical or informatics-based research where a virtual or hybrid structure for research activities and mentorship may be sufficient—and comparable—to in-person.

In the hybrid (2021) cohort, participants praised the added flexibility and productivity brought about by a balanced curriculum that utilized both virtual and in-person educational formats. Many participants from the in-person (2019) and virtual (2020) cohorts also commented on this fact when asked about what they thought would be potential benefits of a hybrid/virtual format. Keeping this in mind, a hybrid model may be the optimal structure for future cancer research education programs such as SOAR. With some adjustments and added structure (e.g., required in-person events), a hybrid model can preserve the organic interactions, sense of community, and clinical exposure that SOAR participants desire while also maintaining a sense of fluidity and agency in how they and their mentor choose to approach their research project and mentor–mentee relationship.

Even with the forced evolution of the SOAR educational format due to the COVID-19 pandemic, the qualitative data illustrates the value of cancer research education programs such as SOAR. Regardless of format, the results reported in this study demonstrate that participants across all cohorts found their research experience educational and fulfilling, engaged with the SOAR seminar series, learned about the multidisciplinary and interprofessional nature of cancer care and the providers involved, and shaped their future career exploration based on their experience in the SOAR program.

It is imperative that programs like SOAR continue to evolve and adapt to both changes brought on by the pandemic and society in general. There is a shift from the traditional physician-scientist paradigm to a “physician-investigator” model that encompasses many types of researchers [15]. The results reported here demonstrate that cancer research learners have different needs that are encompassed across all the SOAR curriculum formats. Cancer research education programs need to evolve away from a “one-size-fits-all” approach to academic medicine and research and create programs that are individualized to the budding researcher [16]. This will help to maintain a strong pipeline of cancer researchers in the future.

This study has some limitations. The SOAR program could be subject to selection bias, with participants coming into the program with a strong interest in oncology, which may lead to biased survey and interview data. Additionally, the survey and interview data could be subject to a social desirability bias, with participants answering questions according to what they think program facilitators want rather than with their true opinions. Finally, a primary limitation of the interviews is that they were done retrospectively, which may lead to recall bias.

In conclusion, this study demonstrates the efficacy of all three SOAR curriculum formats—in-person, virtual, and hybrid—with subjective improvement in the understanding of oncology as a clinical and research discipline in all three cohorts. Qualitative data demonstrates that a completely in-person curriculum may lack flexibility, while a completely virtual curriculum can be isolating in terms of interaction and experience. While the quantitative data suggests that there was broadly no change in outcomes between cohorts, the qualitative feedback illustrates the value of in-person experiences such as interprofessional shadowing. More research needs to be done to delve into the specific benefits afforded by the different components of each educational format. The optimal curriculum format may lie in the middle, with a hybrid structure. Regardless, if research program facilitators remain steadfast in their approach to delivering cancer research and education curricula, programs such as SOAR can remain robust and effective in all educational formats, allowing for the delivery of an innovative cancer research education experience that will nurture the next generation of cancer researchers and ultimately benefit future patients.

Supplementary Information

Below is the link to the electronic supplementary material.

Funding

This study was funded in part by NIH 1R25CA240134-01, NIH CTSA UL1 TR000430, and the Pritzker School of Medicine.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Declarations

Conflict of interest

Vayani, Asif, Klein, Hahn, Pearson, Arora, and Olopade: None. Golden: Grant funding from the National Institutes of Health and Radiation Oncology Institute, Manager of RadOncQuestions LLC.