The Implementation and Effectiveness of PathElective.com

Abstract

The COVID-19 pandemic put most in-person pathology electives on-hold as departments adapted to changes in education and patient care. To address the subsequent void in pathology education, we created a free, virtual, modular, and high-quality pathology elective website. Website traffic from June 1, 2020, to October 1, 2020, was monitored using the built-in analyses on Squarespace. Twitter engagement was analyzed using Twitter analytics and the Symplur Social Graph Score. A voluntary satisfaction survey was sent to all PathElective users and results were analyzed. During this time, the site saw 25 467 unique visitors, over 34 988 visits, 181 302 page views, and 4449 subscriptions from 99 countries. Countries with the highest traffic are the United States (14 682), India (5210), and the Philippines (2195). PathElective’s Twitter social graph score increased from 63.59 to 89.3 with the addition of 1637 followers. Data from surveyed users (n = 177) show most to be pathology residents (41%). Most subscribers (89%) are committed to a career in pathology. The majority heard of the website via Twitter (55%). Almost half of those surveyed engaged with the PathTwitter community on Twitter and of those who participated, 99% found that interaction useful. In all survey questions surrounding satisfaction and usefulness, a large majority of the users were either satisfied or very satisfied. PathElective is a novel pathology elective that offers a unique opportunity to educate medical students and residents from around the globe and demonstrates high effectiveness and satisfaction among users.

Introduction

In the US medical education system, pathology is mainly taught as a basic science discipline during the preclinical years.¹ Lack of a required pathology clerkship in clinical years places medical students in the position of either voluntarily seeking out additional pathology exposure through an elective or matriculating without further understanding of pathology and laboratory medicine.² Additionally, pathologists play an increasingly smaller role in integrated course leadership, diminishing student exposure to the field. As a result, few, if any, medical students get adequate exposure to pathology as a career in medicine, leading to a pipeline problem.³ To address this issue, pathology student interest groups and pathology educators across the country encourage both pathology-bound, and non-pathology-bound students to complete pathology electives and/or join pathology social media whenever they can.⁴ These are usually 2- or 4-week rotations that may include anatomic pathology, clinical pathology, both, or subspecialty pathology based on which school is offering it. Some institutions have also experimented with social media–based pathology pedagogy as well.⁵ Many medical students, especially those from osteopathic schools, seek out external rotations wherever pathology is offered.

Pathology electives usually encompass a variety of in-person and hands-on experiences. This allows students to supplement their didactic theoretical pathology education with a real-time understanding of the inner working and day-to-day operation of anatomic and clinical pathology laboratories. Typically, students rotate in the gross room, attend various subspecialty signouts at the microscope, and interact with clinical pathologists in microbiology, chemistry, or blood bank rounds. The COVID-19 pandemic dramatically altered the delivery and implementation of clinical curricula from medical students and residents, and pathology electives fell victim to the social distancing requirements in such a way that most institutions had to cancel in-person elective experiences.⁶

During the early days of the pandemic, the need for a comprehensive, meaningful, and interactive virtual pathology elective became dire. Despite the ability to project scanned slides in a virtual format, most departments did not have the expertise or redundancy to create a complete online experience that could quickly replace in-person electives. As a means of combating stagnation in clinical rotations, PathElective was founded to offer a pathology elective that can be delivered virtually while maintaining high academic standards, a means of tracking success, and the ability to interact with leading pathology faculty from across the country. Herein, we describe the implementation and effectiveness of this elective website and present results of preliminary assessment from hundreds of early users.

Methods

The PathElective website was developed using Squarespace, video lectures were recorded by each professor or sourced from openly available web resources incorporating virtual slides from PathPresenter, reading assignments, or relevant podcasts at the course director’s discretion. All course content was password protected by GoPayWall which required students to sign up for free-to-see content and interact with the course pages. Each course was developed with built-in pre-, and post-lesson quizzes to assess the success of students taking the courses. Additionally, each student receives a certificate of completion upon successful completion of the post-lesson quiz. Every post-lesson quiz also obtained a standard set of Likert scale and open response questions to assess course satisfaction.

Website traffic data were obtained through Squarespace analytics on a daily and weekly basis for the first 3 months of the website being available (June 1, 2020, to October 1, 2020). Geographic and source data were obtained through deidentified IP address analysis built-in to Squarespace analytics. Geographic data were exported directly from the website. Social media influence was analyzed using the Symplur Healthcare Social Graph Scores which are generated weekly and use Twitter engagement, following, and activity with the health care community to generate a cumulative score which is generated weekly. The Healthcare Social Graph Score was analyzed from June 1, 2020, to October 1, 2020, to mirror the Squarespace data.

Direct Twitter engagement was analyzed using Twitter Analytics focused on the date range immediately surrounding the site’s opening to the general public on July 1, 2020: June 25, 2020, to July 25, 2020. This month-long date range indicates the majority of Twitter activity for this account and helps determine the engagement of tweets advertising the opening of various modules as they went live. Twitter analytics, engagement, and impressions were imported to excel, analyzed, and visualized.

An overall satisfaction survey was developed using Google Forms and was emailed to all individuals on the mailing list as well as registered users and the survey was open for 2 weeks. The survey was also published as a pop-up message on the website during the same window to gain additional study participants (n = 177). Participant data were anonymous; however, users were capped at one entry by imposing a Google account limitation. This survey was voluntary, contained Likert-scale, multiple-choice demographic, and website interaction questions.

Results

Within the time frame of this study, the pathelective.com website received 35 000 visits with 25 000 unique visitors that resulted in 181 000 page views. The website was visited by a global audience (Figure 1A) with the highest number of visitors from the United States and India. The site audience was truly global, with visitors from all continents except Antarctica. Within the United States, most users originated from the states of California, Texas, Illinois, and New York (Figure 1B). Within India, while visitors originated from most states, the highest number was from the state of Tamil Nadu (Figure 1C).

Figure 1.

Geographic distribution of PathElective visitors. Heat map analysis of the number of users based on geographic location across the world (A). All continents revealed visits, except Antarctica (not shown). Within the United States (B), the states from which most visitors came included California, Texas, Illinois, and New York. India revealed the second highest number of visitors (after the United States), with most visits originating from the state of Tamil Nadu (C).

Website traffic sources for all 34 516 visits (Figure 2A) included referrals from other sites or blogs, from email clickthrough, search engine result-driven, social media–based, and directly from URL entry. Most users (17 897) visited the site via desktop devices, but mobile users were a close second (17 073; Figure 2B). The site homepage was the most popular, viewed over 30 000 times with the anatomic pathology main page following with over 23 000 visits (Figure 2C). Direct visitors, those who typed the address directly into the URL, accounted for over 26 000 visitors, Google-directed traffic was the next most popular source with almost 4000 visitors being directed from this search engine (Figure 2D). The remaining visitors were directed from Twitter (2785), Facebook (1428), or another source.

Figure 2.

Website traffic and user behavior. Traffic sources ranged from referral to direct website entry (A) that led to the home page being most visited (B). Most users were on desktop devices (C) and entered the site as a result of direct website access (D).

Monitoring of social media metrics from the time of the launch shows impressions on Twitter with engagements mirroring that of the impressions at a fold lower (Figure 3A). The peak Twitter impressions and engagements were 11 000 and 1000, respectively, on July 1, 2020 (site launch). The site launch date is also the date for the highest number of tweets published (Figure 3B). Although the overall impressions, engagement, and number of tweets vary significantly for the remainder of the month, the Symplur Healthcare Social Graph Score demonstrates an incremental increase in this score from May 1, 2020 to October 1, 2020, with sharper increases on June 1, 2020, and July 1, 2020. The first increase (Figure 3C, red arrow) coincides with the time that the site was beta tested with a group of students who completed this elective at Loyola University Chicago Stritch School of Medicine. By the end of the analysis period, PathElective reached a score above 90 which corresponds to the top 9611 profiles of over 6 million total healthcare social media profiles. At this time, the site was closed to outside users and all access was password protected. The second increase on this curve (Figure 3C, blue arrow) reflects the official opening of the site to the international audience. Google search metrics for the time period of our study (Figure 3D) show the highest number of clicks and click-through rate for the term “pathelective” with markedly lower scores for “path elective,” “pathelective.com,” “pathology elective,” “pathelective com,” and “pathelectives.”

Figure 3.

Social media and search performance. Social media performance surrounding the site launch revealed mirroring of Twitter impressions and engagement (A). Twitter activity varied during this time frame, with highest activity surrounding launch time (B). The site’s social media influence over time reveals an incremental increase in Symplur Healthcare Social Graph Score (C) with prominent bursts in early June and early July that correspond with Loyola University Chicago SSOM beta testing start date (red arrow) and the PathElective public release date (blue arrow). An analysis of Google search metrics reveals a reasonably consistent clickthrough rate for the top 6 search terms presented (D).

Site users who responded to the survey revealed a predominance of US pathology residents (41%) followed by practicing physicians (20%), with international medical graduates comprising 17% and US MD/DO students comprising 13% of all responders (Figure 4A). As a result, when asked about their level of interest in pathology, the majority of respondents (57%) answered that they were either currently in practice or in pathology training programs. Approximately one-third (33%) of the survey responders were interested in, and committed to pathology, while 8% were interested in pathology, but not fully committed at the time of completing the survey. The vast majority of the respondents found the website from information on Twitter (56%), followed by word of mouth (16%) and Instagram (11%; Figure 4C).

Figure 4.

Survey demographics and course engagement. Summary of survey results including demographics (A-C) and course engagement (D-G).

When asked about course engagement, the majority of survey respondents engaged in between 1 and 3 courses, and this majority was also seen in the number of certificates received as well. Fewer respondents said they engaged in >6 courses and the least number of respondents engaged in between 4 and 6 courses (Figure 4D). When looked at in further detail, course engagement was skewed toward anatomic pathology (nearly 160 responses) with fewer engagements in clinical pathology models (80 responses; Figure 4E). Within the AP and CP groups, gross pathology and hematology were the most frequently visited courses, respectively (Figure 4F).

Survey questions related to user satisfaction inquired about several features on the site. When asked about the user’s overall satisfaction with the site (Figure 5A) and the quality of the content on the site (Figure 5B), the responses to both questions were similar, with the majority of respondents stating they were 5, or 4 on a 1 to 5 Likert scale, where 5 indicated very satisfied. The spread of the responses was similar when asked about the functionality of the website (Figure 5C) with most survey respondents (over 100), indicating they were very satisfied (5/5). When asked whether PathElective users felt that their interaction with the website helped advance their training, or improve their daily practice (Figure 5D), the answers indicated mostly 5 or 4 on the Likert scale (where 5 indicated the respondent strongly agreed). Similar responses were noted in response to questions on whether the respondents could see pathelective.com being a supplement to in-person elective (Figure 5E), and if the platform was helpful for their career given the cancellation of in-person electives (Figure 5F)—responses to both questions demonstrated an overwhelmingly positive response of 4 or 5 (where 5 was most positive). When asked if virtual slide training sessions, such as “unknown cases” would be taken advantage of, if offered, most respondents thought they would benefit from it (Figure 5G).

Figure 5.

Response to satisfaction survey. Overall satisfaction metrics. These data are based on Likert scale responses from participants with 1 to 5 representing very unsatisfied, unsatisfied, neither satisfied nor dissatisfied, satisfied, and very satisfied, respectively (A-C), or strongly disagree, disagree, neither agree nor disagree, agree and strongly agree, respectively (D-G).

Discussion

PathElective.com is the first virtual pathology elective forum with extensive and in-depth coverage of several subspecialty disciplines with pathology. An internationally renowned group of pathologists and mentors put forward their contributions to this site. Virtually, module faculty guide the user through various areas of pathology by a series of videos, objectives, and resources as they complete their elective online. A large number of visits from the United States and international countries to the PathElective.com website within such a short period of time since its inception is a testament to the potential power of reach this readily accessible resource provides to medical students and pathology trainees. We had initially envisioned pathelective.com to be most useful to medical students; however, given a large number of visits by pathology residents and trainees we believe that the website provides a valuable resource to pathology trainees, particularly those in their earlier stages of training who are familiarizing themselves to basic pathology concepts. Overall, the results of the survey questions revealed our users to be satisfied with the quality of content on the website as well as the functionality but the most encouraging result of the survey was the positive feedback indicating that the users believed patheletive.com content helped advance and improve their daily practice. We believe that this pilot project provides evidence for the potential global reach and abilities of such educational resources for various areas of continuing medical education across all subspecialties in pathology and beyond.

As educators, the authors had long realized the need for a platform from which to disseminate exposure to pathology education. The idea of the pathelective.com website (formulated by site founder and coauthor K.M.M.) was brought forward to his medical student (cofounder and coauthor C.M.L.) in April 2020 as the COVID-19 pandemic began to affect medical rotations in the United States. The name “pathelective” was selected for this project, and a group of additional content experts (all cofounders and current coauthors on this article) were invited to participate in further brainstorming ideas for the PathElective online platform. An initial brainstorming session for the cofounders was held on April 13, 2020, via the Zoom videoconferencing platform. Various ideas were proposed and discussed regarding the format of the PathElective platform. The main concept was that the PathElective website would host a variety of subspecialty-specific pathology elective modules and that the creation of these modules would be overseen by faculty with expertise in these areas. It was decided that the site will be free for all users. A skeleton outline for the general format of the modules was formulated. After this meeting, work on several core modules began, including creating and organizing content and developing pre- and post-test questions. The website was designed, built, and tested to ensure that the modules could be easily accessed and completed by online users. A Twitter account was created for the project (@PathElective) in April 2020. In May and June 2020 (prior to the July 1, 2020, go-live date), announcements and promotion of the upcoming pathelective.com website were tweeted by the @PathElective account and by the Twitter accounts of many of the cofounders. Pathelective.com was also promoted on other social media platforms including Facebook, Instagram, and YouTube. This promotion continued after the go-live date. There are over 7000 pathologists/pathology-related accounts on Twitter alone.^7,8 The #PathTwitter community and pathology communities on other social media platforms are very active and are used by many pathologists worldwide.^9-11 Additionally, many of the PathElective cofounders are highly active social media users with large online followings. The cofounders were not merely online promoters of pathelective.com; they were also content creators who helped design and build it.¹² Social media–based distribution of information about pathelective.com proved to be a very powerful method of raising awareness and likely contributed greatly to the high volume of early adopters who signed up for and used pathelective.com within days and weeks following the go-live date.

The virtual platform had many advantages, which have applicability beyond the need to socially distance.⁶ The free and asynchronous nature allowed worldwide learner participation by breaking the traditional barriers of finance, distance, and time zones. The on-demand nature of the materials allowed for learners to study the content at their own pace, on their own schedule, and all for free.¹³ Another advantage is the wide range of expert faculty from across the country and across institutions, whose combined subject matter expertise and teaching experience make for robust content curation and creation. Virtual slides and digital pathology resources translate the traditional slide box–based “teaching set” model to a worldwide audience, allowing the sharing of educational cases in a manner similar to seeing cases under the microscope. The certificate format allowed participants to demonstrate their participation, which is an important element, as this allows for documentation of interest and education obtained. Although the lack of in-person rotations initially inspired this platform, the reach and ease of access of this virtual platform will continue its impact beyond the pandemic.

PathElective.com strived to create an interactive experience, with a blend of video didactics, whole slide images for independent review, and questions with explanations. Although the user experience is modeled after teaching and independent case review that takes place during in-person electives, virtual experiences cannot fully replicate the highly interactive discussions that students can engage in during the management of live clinical cases. We believe that our user satisfaction reflects the educational value of the modules developed for PathElective.com; however, medical students and residents participating in elective rotations often have additional goals beyond gaining knowledge. In-person rotations are opportunities for medical students and residents to evaluate the suitability of the location for their future training, as well as seeking letters of recommendation for residency and fellowship applications. These aspects of an elective experience cannot be replicated with a virtual elective, and without these factors, some institutions do not formally recognize virtual electives.

At the time of its implementation, the primary goal of this modular website was to expose students to pathology, especially to those who could not pursue electives in their own institutions. However, a broader and deeper utility of the site for more formal educational experiences (such as official institution elective) was not explored. Anecdotal data show that several dozen students listed these elective courses as “miscellaneous” pathology experiences in their Electronic Residency Application Service (ERAS) applications for the 2021 pathology Match. It is unknown whether this enhanced their residency application in any constructive way. The authors had considered this possibility at the time of site creation and had implemented a “for educators” tab on the site to provide faculty an understanding of the value of a certificate. However, beyond this simple benefit, the certificates provided for completed modules do not provide more insight into the students’ experience. Right now, the certificates or the site do not have a Continuing Medical Education (CME) component or provide direct elective credit; however, the authors are actively working on this possibility. To date, we know of select institutions that have implemented pathelective.com modules as a component of their institutional elective experience (eg, Loyola University Chicago Stritch School of Medicine PATH-415 elective course) with great success. Our aim is to expand this use of the website as an adjunct to in-person pathology learning.

Social media provided an important outlet for virtually celebrating medical school graduation for the class of 2020, and the pathology match (#virtualpathmatch). As the pandemic continued to wreak havoc across the country, pathelective.com provided an additional opportunity to the medical school graduates of 2021 applying for the pathology match (#PathMatch21).¹⁴ Educators and pathology candidates alike were able to use the certificates generated by the PathElective website to gauge and show interest in pathology, respectively. Anecdotally, several interviewing candidates mentioned the pathelective.com website as a unique aspect of the interview season in fall 2020 and mentioned the benefit it provided to their education.

The utility and success of a pathology elective varies across medical institutions depending on the enthusiasm, teaching expertise, and schedule of experiences each department allows. This, in turn, is confounded by how much faculty have to teach and their specific expertise. Teachers on busy clinical services, despite best intentions, may not have time to spare, which allows for students to have a less wholesome experience. In other instances, based on which subspecialties are strongly represented (or not), learners may have a biased exposure to only anatomic or clinical pathology and not experience the full breadth and depth of the pathology and laboratory medicine experience. Using pathelective.com as a curricular adjunct allows the training algorithm to equalize exposure to all aspects of anatomic, clinical, and molecular pathology uniformly, and across the world.