Abstract
Since 2018, we have evaluated the effectiveness of various teaching technologies for training young investigators on translational research in cancer health disparities. The Southeast Partnership for Improving Research and Training in Cancer Health Disparities (SPIRIT-CHD) unites Moffitt Cancer Center and the Louisiana State University Health Sciences Center. One of the main components of the SPIRIT-CHD is the Cancer Research Education Program (CREP) for training undergraduate and medical students from underrepresented backgrounds. The CREP utilizes a web-based didactic curriculum to engage students at both institutions in biobanking, precision medicine, and cancer health disparities topics. We report experiences from our cross-institutional cancer education program, specifically evaluating the cohorts’ satisfaction and learning gains using various communication technologies and instructional approaches. Trainees completed a survey with questions evaluating the curriculum and technology. Trainees reported satisfaction with the flipped classroom model (FCM) content and overall program (mean score = 3.2, SD = 0.79), and would recommend the program to peers. Yet, despite improved program delivery, trainees felt interaction between the two sites (mean score = 1.5, SD = 0.85) and engagement with faculty (mean score = 2.80, SD = 1.14) could be improved. The technology with the highest reported use was e-mail, with a mean score of 4.6 (SD = 0.52). LinkedIn and Twitter had the lowest frequency of use with mean scores at 1.90 (SD = 0.99) and 1.30 (SD = 1.34). Our study highlights the successes and challenges of remote learning using technology to increase interaction and engagement among trainees and faculty in a multi-site cancer research training program.
Keywords: Cancer disparities, Biobanking, Precision medicine, Cancer education program, Technology, Underrepresented trainees, Flipped classroom
Introduction
Cancer health disparities are often accentuated by intersecting racial, ethnic, sexual, and gender identities, whereby underrepresented minority groups may have limited access to participate in precision medicine studies and opportunities to contribute to biobanking/biospecimen research [1–3]. The lack of participation from underrepresented groups creates both gaps in cancer research and limits generalizability of findings to populations that were not part of the original research. There is a national priority to train underrepresented biomedical researchers to conduct health disparities research and education in medically underserved/understudied communities as a way to address these gaps in precision medicine and biospecimen-based science [4]. Diversifying the biomedical science workforce to include scientists from underrepresented groups may improve community trust in research, provide fresh perspectives on research approaches, and enhance the development of culturally inclusive educational programs [5]. The primary purpose of this manuscript is to report on experiences from the Southeast Partnership for Improving Research and Training in Cancer Health Disparities (SPIRIT-CHD), a joint cross-institutional cancer education program designed to foster interest in cancer health disparities research, precision medicine, and biobanking/ biospecimen science research among underrepresented undergraduate and medical students. Specifically, we evaluated changes made to communication technologies for remote learning and instructional approaches from our first cohort to our second cohort, based on learner feedback.
The SPIRIT-CHD consists of the H. Lee Moffitt Cancer Center and Research Institute (MCC), a National Cancer Institute-designated comprehensive cancer center located in Tampa, Florida, and the Louisiana State University Health Sciences Center (LSUHSC), an institution serving underrepresented populations in New Orleans, Louisiana. The cross-institutional collaboration goals are to (1) advance translational research on the biological mechanisms of cancer health disparities; and (2) create a Cancer Research Education Program (CREP) for undergraduate and medical trainees from underrepresented backgrounds. The goal of the CREP is to increase diversity in the biomedical research workforce by training the next generation of cancer researchers with a specific focus on biobanking, precision medicine, and cancer health disparities.
The CREP has three main components: (1) in-person research experiences; (2) community outreach and engagement activities; and (3) a web-based didactic curriculum: ARISE-CHD (biobAnking, pRecISion mEdicine and Cancer Health Disparities). Because the CREP is a multi-site program, pedagogic approaches that foster interaction and engagement are needed to connect trainees within and across institutions. This manuscript describes the 8-week ARISE-CHD didactic summer curriculum and its enhancement of technology use from the first to second cohort and learner satisfaction from underrepresented trainees’ cross-institutional engagement at two partnering institutions (MCC and LSUHSC).
Methods
The ARISE-CHD Curriculum
The curriculum is composed of eight didactic sessions on the following topics: Biobanking, Bioethics of Biobanking, Cancer Health Disparities, Culture and Health Literacy, Community Outreach and Engagement, Precision Medicine, and Responsible Conduct of Research. For the first 2018 cohort (n = 8), the sessions were delivered using traditional teaching methods. Each week, the faculty presenter lectured to LSUHSC and MCC trainees through web-based video-conferencing software (Zoom) on a specified topic for 2 h. Initial post evaluation results from cohort 1 revealed at the majority felt the length of the sessions was too long and requested increased interaction and engagement between faculty and trainees, cross-institutionally.
“The 2-hour sessions are much too long…. using [Zoom]; no one gets anything from it.”
“I would’ve loved to interact with the other students more throughout the program”
“….there should have been more interactions with the (faculty)... It would have been helpful to get to know them more”
In response to these recommendations, operational changes were made to the ARISE-CHD curriculum for the second cohort (2019) to shorten each lecture’s duration and increase interaction and engagement among trainees and faculty by implementing the flipped classroom method (FCM) (Fig. 1). At least 1 week before each scheduled session, learning content was uploaded to Slack, a multipurpose technology platform. Trainees were instructed to review the content before the lecture. The weekly sessions elaborated on discussion questions; created interactive polls; and collectively deliberated case studies. To further increase interaction and engagement, each week one student from LSUHSC and one student from MCC were chosen to work with faculty in co-facilitating a didactic session. The FCM was piloted with four of the eight didactic sessions: Responsible Conduct in Research, Precision Medicine, Cancer Health Disparities, and Culture and Health Literacy. The other four didactic sessions were presented in the traditional method previously described.
Fig. 1.
Transition to the flipped classroom method (FCM) model
FCM is an emerging active pedagogical approach to active learning and one of the most popular “technology-infused” learning models. It is an instructional strategy, and blended learning approach focused on student engagement and active learning [6]. The FCM involved students watching online lectures, collaborating in online discussions, and reviewing material on their own time before engaging in the virtual classroom. The FCM works in alignment with digital technology platforms such as Slack, Google Drive, and Zoom, fostering student engagement in pre-class learning (i.e., instructional videos, interactive assignments, readings), allowing class time to build on essential topics and demonstrate the application of key concepts [4, 7]. Positive student engagement is often seen due to unrestricted access to technological resources (i.e., Slack, Zoom, Microsoft Outlook e-mail) to prepare for class, and more in-class active learning to increase trainees’ understanding of the subject material [8].
Technologies and Social Media Used in ARISE-CHD
At each institution, trainees were provided an e-mail through Microsoft Outlook to communicate with their lab team, mentor, and peers. Slack was the repository for all ARISE-CHD content. Trainees utilized Google Drive to plan co-facilitation roles and organize information. Each week trainees connected cross-institutionally for their joint didactic sessions using the video-conferencing application Zoom. While on Zoom, one faculty member used the interactive polling software TurningPoint to ask questions and obtain real-time responses.
LinkedIn and Twitter were two additional social media applications used. LinkedIn was used as a professional career development tool to expand trainees’ professional network and establish connections by building an online resume. The SPIRIT-CHD also uses LinkedIn to track trainees’ future professional accomplishments. Twitter, a social media application that allows individuals to share short microblogging messages called “tweets,” was used to encourage trainees to participate in guided tweet conversations focusing on a specific topic or hashtag related to cancer health disparities, precision medicine, or biobanking.
Assessments
Trainees completed demographic information and a post-assessment of satisfaction with the ARISE-CHD curriculum content and technology. The survey contained four questions: (1) understanding of content; (2) overall satisfaction with the program; (3) faculty engagement; and (4) student interaction. The nine technology applications were evaluated regarding the usefulness, accessibility, frequency, ease of use, and overall satisfaction with technology. Responses were based on a 6-point scale (0 = not applicable; 1 =never; 2 =almost never; 3 = sometimes; 4 = frequently; 5 =very frequently) (see Table 1). In addition, three open-ended response questions asked trainees to elaborate on their responses to the quantitative items.
Table 1.
Evaluation of technology applications
| n = 10 | Frequency of use* | Ease of use* | Accessibility* | Usefulness* | Satisfaction* | Most used | Least used | Most useful | Least useful |
|---|---|---|---|---|---|---|---|---|---|
|
| |||||||||
| Slack | 3.40 (0.97) | 4.00 (0.94) | 4.10 (1.10) | 3.50 (1.07) | 3.60 (0.97) | - | - | 1 (10%) | - |
| 1.90 (0.99) | 3.10 (1.91) | 3.50 (2.07) | 2.30 (1.70) | 2.90 (1.85) | - | 2 (20%) | - | 2 (20%) | |
| 1.30 (1.34) | 1.40 (1.96) | 1.90 (2.28) | 0.90 (1.52) | 1.30 (1.89) | 1 (10%) | 3 (30%) | - | 4 (40%) | |
| 4.90 (0.32) | 4.60 (0.52) | 4.60 (0.52) | 4.90 (0.32) | 4.20 (0.42) | 8 (80%) | 1 (10%) | 5 (50%) | - | |
| Google Drive | 3.10 (1.66) | 3.90 (1.73) | 3.90 (1.85) | 3.80 (1.62) | 3.90 (1.66) | 1 (10%) | - | 4 (40%) | 1 (10%) |
| Zoom | 3.10 (1.37) | 3.50 (1.43) | 3.50 (1.65) | 3.40 (1.26) | 3.60 (1.17) | - | - | - | 1 (10%) |
| Turning Point | 2.90 (1.20) | 3.50 (1.43) | 3.50 (1.42) | 3.00 (1.63) | 3.60 (1.57) | - | 1 (10%) | - | 2 (20%) |
Rated on a scale of 0–5
Data Analysis
Descriptive statistics of evaluation results are reported for the second cohort (2019–2020). Open-ended responses were analyzed using content analysis. Direct quotes were de-identified and are provided as illustrations. This project was reviewed, approved, and deemed exempt by the Institutional Review Boards of MCC and LSUHSC.
Results
Demographics
Cohort 1 was composed of four undergraduates and four medical students enrolled in the Cancer Health Education Program (CREP) at LSUHSC or MCC in 2018. Cohort 2 was composed of eight undergraduates and four medical students in 2019. The second cohort data are reported to capture the transition into the FCM. In the second cohort, the majority of trainees were female (n = 9; 81.8%). African American trainees comprised 66.7% (n = 8) and 33.3% (n = 4) identified as Hispanic and/or Latinx. Approximately 72.0% (n = 8) reported their sexual orientation as straight, 9.1% (n = 1) as gay/ lesbian, and 18.2% (n = 2) as other.
Satisfaction with the ARISE-CHD curriculum
Trainees rated their understanding of the content in the FCM with a moderate comprehension mean score of 3.4 (SD = 0.52), and were moderately satisfied with the content, with a mean score of 3.2 (SD = 0.79). Engagement between students and faculty in the ARISE-CHD program had a low satisfaction mean score of 2.80 (SD = 1.14). Student ratings of interactions between MCC and LSUHSC students during the ARISE-CHD program were low, with a mean score of 1.5 (SD = 0.85).
As noted in Table 1, students provided evaluations of the frequency of use, ease of use, accessibility, usefulness, and satisfaction with technology implemented during the ARISE-CHD. The technology with the highest reported use was e-mail, with a mean score of 4.6 (SD = 0.52). Google Drive, Zoom, Slack, and TurningPoint were “sometimes” used by trainees with average scores of 3.10 (SD = 1.66), 3.10 ( SD = 1.37), 3.40 (SD = 0.97), and 2.9 (SD = 1.20), respectively. LinkedIn and Twitter had the lowest frequency of use, with mean scores at 1.90 (SD = 0.99) and 1.30 (SD = 1.34). Trainees were moderately satisfied using Google Drive, Zoom, and Slack with mean scores of 3.90 (SD =1.66), 3.60 (SD = 1.17), and 3.60 (SD = 0.97), respectively. Regarding technology satisfaction, trainees were somewhat satisfied with TurningPoint, with a mean score of 3.30 (SD = 1.57). Trainees had lower satisfaction with LinkedIn, as reflected by a mean score of 2.90 (SD = 1.85), and Twitter had the lowest satisfaction score with a mean of 1.30 (SD = 1.89). See Table 1 for additional information.
Among trainees who provided a written response to the open-ended question about satisfaction with ARISE-CHD, the majority made positive statements and provided recommendations for improvement.
“Everything was great!”
“This part worked well, the only problem with Slack was that the audio file did not work for one of the lectures I tried to download from it.”
“My suggestion would be to utilize Slack more. Discussion prompts could be put up for each module to allow (all trainees) to reply and discuss on Slack,”
Trainees were asked to provide feedback regarding how the interaction could be improved. The majority noted some problems with the technical aspects of the technology and suggested ways to strengthen interactions between the two sites.
“The Zoom meetings were not effective, and I did not feel engaged with the other students. Group chats between the two groups… (would) provide updates on what the other location is doing.”
“… more incorporation of SLACK would be better. Using it to actually have assigned blog post discussions before the seminars, where you have to respond to at least two people’s posts one from MCC and one from LSU. Actually, having a deep-dive partner for each seminar. Having a “buddy” that you’re responsible for getting to know.”
“Perhaps discussing the material in Slack on a discussion board prior to meetings”
Discussion
Our study reported on the experiences from a cross-institutional summer cancer education program and highlighted the use of various communication technologies. Our results showed moderate satisfaction with the FCM despite being a common approach among health professional educators [8, 9]. Recent studies suggest the FCM yields a significant improvement in student learning compared to traditional teaching methods [8, 9]. A meta-analysis of 28 comparative studies showed an overall significant effect in favor of FCMs over traditional teaching methods for health professions education [8]. However, limited evidence exists regarding the utilization of the FCM to engage underrepresented trainees in active learning and dynamic interaction [10] in cancer and cancer health disparities research.
Despite the improvement made to ARISE-CHD instruction delivery, trainees felt interaction and engagement between the two sites were overall low and could be further improved. These modifications were designed to allow trainees to demonstrate peer-to-peer learning through their in-session interactions. However, this may not have been sufficiently implemented to be regarded as fully interactive and engaging.
Each technology and social media platform used for ARISE-CHD was evaluated for the frequency of use, ease of use, accessibility, usefulness, and satisfaction. Results show that specific platforms, namely, Microsoft Outlook e-mail, received consistently higher evaluation scores, while others, such as Twitter, ranked last. For example, Microsoft Outlook e-mail was used on an almost daily basis for communications and calendar appointments between trainees and faculty. Slack, another high-scoring platform, was utilized weekly for its discussion board features and sharing required session materials. Slack has shown success in increasing student engagement [11].
On the other hand, Zoom was used solely for video-conferencing classroom purposes. Furthermore, the crowd polling software TurningPoint was only used during one faculty lecture. Trainees were strongly encouraged to build their professional resume online using LinkedIn—however, it was not enforced as a requirement for successfully completing the CREP program. Similar to previous studies [12], Twitter was intended to be used for live chats and discussions surrounding relevant curriculum topics; however, these activities did not take place as intended.
Although the current study does not make direct comparisons between traditional and web-based modules, other studies have shown that web-based modules can lead to higher knowledge scores [13]. E-learning has also been shown to capture content effectively while offering student flexibility; however, these advantages can be offset when content and assignments are overloaded within a limited time frame, as illustrated by ARISE-CHD [14]. Nevertheless, the incorporation of a multiple content delivery platforms (i.e., Twitter, Zoom, Slack) follows the overarching trend towards “highly interactive self-directed learning environments” (p. 2285) in medical education [15]. Alternatively, fusing the FCM with educational technologies and learning strategies such as augmented reality (AR)-based learning could help improve ARISE-CHD trainee satisfaction scores with the FCM and project performance [16].
Trainees’ comfort level using specific social media platforms in educational settings could have impacted the evaluation. Trainees who use Twitter for social purposes in their free time may not want to use their personal Twitter account for academic purposes. Trainees who were not as familiar with Twitter may have also felt uncomfortable using the platform. Anecdotal reports suggest faculty did not model the use of these technologies as they typically interacted with trainees using Microsoft Outlook e-mail. In contrast, use of multipurpose technology applications such as Slack, which offers group and direct one-on-one message and multimedia file sharing, could potentially enhance interactions. Increasing the frequency with which trainees interact through platforms by scheduling weekly live-chat sessions on Twitter or having trainees reflect on specific program experiences (i.e., presenting research, visiting a biobank) via LinkedIn could improve satisfaction with and use of technologies. Alternatively, focusing on selected technologies that are productive and frequently used by both faculty and students could also improve outcomes.
This paper is one of the first studies reporting the use of technologies among diverse, underrepresented undergraduate and medical student trainees in a multi-institutional cancer health disparities research training program. Despite this, the study has limitations. First, we were unable to compare actual satisfaction results between cohort 1 and cohort 2 because we did not use the same assessment measures. Second, our sample size was small, and the results may not be generalizable to other training programs. Third, the FCM was not used in every session. In the future, we will assess satisfaction for both FCM and traditional formats to evaluate differences. Finally, responses may be subject to social desirability bias.
Conclusion
The CREP program offers a unique opportunity for underrepresented undergraduate and medical student trainees in two different geographic locales to be immersed in cancer health disparities research. Moreover, trainees took part in community outreach and engagement experiences in underserved communities, and participated in ARISE-CHD, a web-based didactic curriculum focusing on biobanking, precision medicine, and cancer health disparities. The results of our study highlight the successes and challenges of using diverse communication technologies and innovative pedagogic approaches to increase interaction and engagement among trainees and faculty participating in a multi-site cancer research training program. Given the explosive proliferation of communication and instructional technologies, mixed approaches may help with feasibility as we enter a new culture of learning while simultaneously leveraging multi-use technology applications in interprofessional education.
Funding
Funding for the project was provided by grant P20 CA20292001A1, Southeast Partnership for Improving Research and Training in Cancer Health Disparities (SPIRIT-CHD) (Meade and Miele, PIs) funded by the National Cancer Institute. The efforts of Dr. Cousin were supported by grant #5T32CA090314–16 (PI: T. H. Brandon) while she was a postdoctoral fellow. The funders had no role in the study design, collection, analysis, and interpretation of data, the writing of the manuscript, or the decision to submit the manuscript for publication.
References
- 1.Siegel RL, Miller KD, Jemal A (2019) Cancer statistics, 2019. CA Cancer J Clin 69(1):7–34 [DOI] [PubMed] [Google Scholar]
- 2.Spratt DE, Chan T, Waldron L, Speers C, Feng FY, Ogunwobi OO, Osborne JR (2016) Racial/ethnic disparities in genomic sequencing. JAMA Oncol 2(8):1070–1074 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.National Cancer Institute (2019) Cancer Disparities. [cited 2019 2/26/2020]; Available from: https://www.cancer.gov/aboutcancer/understanding/disparities
- 4.National Institutes of Health (2011) The National Academies Collection: reports funded by National Institutes of Health. In: Lorden JF, Kuh CV, Voytuk JA (eds) Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. National Academies Press (US), Washington (DC) [PubMed] [Google Scholar]
- 5.Simon MA, de la Riva EE, Bergan R, Norbeck C, McKoy JM, Kulesza P, Dong XQ, Schink J, Fleisher L (2014) Improving diversity in cancer research trials: the story of the Cancer Disparities Research Network. J Cancer Educ 29(2):366–374 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Abeysekera L, Dawson P (2015) Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. High Educ Res Dev 34(1):1–14 [Google Scholar]
- 7.National Institutes of Health (2017) What is precision medicine? 2/26/2020; Available from: https://ghr.nlm.nih.gov/primer/precisionmedicine/definition
- 8.Hew KF, Lo CK (2018) Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ 18(1):38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Jensen JL, Kummer TA, Godoy PD d M (2015) Improvements from a flipped classroom may simply be the fruits of active learning. CBE Life Sci Educ 14(1):ar5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Halpern MT, Dodd SJ, Fang CY, Tan Y, Zhu L, Ogunwobi OO, Ma GX (2019) Evaluation of a transdisciplinary cancer research training program for under-represented minority students. Inf Sci IT Educ Conf 2019:99–108 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Cook J, Mabe R, Harman B (2019) An exploration into the use of the digital platform Slack to support group assessments and feedback and the impact on engagement [Google Scholar]
- 12.Williams D, Whiting A (2016) Exploring the relationship between student engagement, Twitter, and a learning management system: a study of undergraduate marketing students. Int J Teach Learn High Educ 28(3):302–313 [Google Scholar]
- 13.El-Ali A et al. (2019) Comparison of traditional and web-based medical student teaching by radiology residents. J Am Coll Radiol 16(4):492–495 [DOI] [PubMed] [Google Scholar]
- 14.Premkumar K et al. (2010) Technology-enhanced learning of community health in undergraduate medical education. Can J Publ Health 101(2):165–170 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Zafar S, Safdar S, Zafar AN (2014) Evaluation of use of e-learning in undergraduate radiology education: a review. Eur J Radiol 83(12):2277–2287 [DOI] [PubMed] [Google Scholar]
- 16.Chang S-C, Hwang G-J (2018) Impacts of an augmented reality-based flipped learning guiding approach on students’ scientific project performance and perceptions. Comput Educ 125:226–239 [Google Scholar]