Abstract
Objectives
The objective was to bridge the relative educational gap for newly matched emergency medicine preinterns between Match Day and the start of internship by implementing an Accreditation Council for Graduate Medical Education Milestone (ACGME)‐based virtual case curriculum over the social media platform Slack.
Methods
We designed a Milestone‐based curriculum of 10 emergency department clinical cases and used Slack to implement it. An instructor was appointed for each participating institution to lead the discussion and encourage collaboration among preinterns. Pre‐ and postcurriculum surveys utilized 20 statements adapted from the eight applicable Milestones to measure the evolution of preintern self‐reported perceived preparedness (PP) as well as actual clinical knowledge (CK) performance on a case‐based examination.
Results
A total of 11 institutions collaborated and 151 preinterns were contacted, 127 of whom participated. After participating in the Slack intern curriculum (SIC), preinterns reported significant improvements in PP regarding multiple Milestone topics. They also showed improved CK regarding the airway management Milestone based on examination performance.
Conclusions
Implementation of our SIC may ease the difficult transition between medical school and internship for emergency medicine preinterns. Residency leadership and medical school faculty will benefit from knowledge of preintern PP, specifically of their perceived strengths and weaknesses, because this information can guide curricular focus at the end of medical school and beginning of internship. Limitations of this study include variable participation and a high attrition rate. Further studies will address the utility of such a virtual curriculum for preinterns and for rotating medical students who have been displaced from clinical rotations during the novel coronavirus pandemic.
Transitioning from medical school to postgraduate training in emergency medicine (EM) is challenging. There are relatively few educational experiences between Match Day and the start of internship. Also, the clinical experiences and medical school curricula of these soon‐to‐be graduating medical students (whom we term “preinterns”) vary considerably. 1 The standard recommendation for the EM applicant is to complete EM subinternships by fall of the application year. 2 While this timeline is prudent for the application process, it follows that for most preinterns, the last EM rotation is 6 months or more prior to the start of internship.
Because of the differences in EM educational checkpoints used by undergraduate and graduate medical education institutions, the transition from medical school to internship can vary greatly between both individual preinterns and residency programs, with little time to adjust to the new clinical demands of internship. 3 Prior investigations have evaluated pregraduation or early‐internship boot camps to ease this transition, 4 , 5 , 6 but there are few previously described curricula for the time between Match Day and the start of internship. 7 Prior standardized evaluations of graduating medical student skills have utilized transitional year Accreditation Council for Graduate Medical Education (ACGME) Milestones. 8 However, there has been no prior assessment of EM‐bound preinterns utilizing the EM ACGME Milestones in Patient Care 9 (which we simply term “Milestones”) for preparedness or clinical knowledge (CK) assessment.
An exhaustive literature search did not reveal any social media intervention addressing the transition from medical school to EM internship. Specifically, we completed a Medline advanced search using the OVID interface for the year 2000 through 2020, with a limit to English language works. We searched for the phrase “social media,” combined with the phrase “medical education” in the abstract fields, which yielded more than 100 results. This list and many of the publications cited by these works were reviewed, and those most representative of our subject were incorporated into our background information. To our knowledge, there is also no existing curriculum designed for this time period. Our literature search did reveal one recent study utilizing social media to administer an organized medical education intervention during residency. 10 Messman et al. 10 used Slack 11 to teach 36 EM residents to read diagnostic images in a single residency program. Of the resident survey respondents, 100% reported that the curriculum was beneficial, and 68% reported a substantially improved overall comfort level with independently reviewing diagnostic imaging. Objectively, resident confidence in image interpretation increased significantly from 2 to 3.5 on a Likert scale of 1 to 5.
Prior work from our own project group has been published. A prospective observational pilot study utilizing a preliminary version of our Slack intern curriculum (SIC) was implemented for nine preinterns at one institution in spring of 2017. 12 Data from this pilot suggested that preinterns felt more comfortable entering intern year after completing the curriculum (mean overall comfort level increased from 3.1 pre‐SIC to 3.7 post‐SIC on a Likert scale of 1–5). Preinterns praised the ease of use, diversity of topics, and ability to participate remotely. They cited the overall helpfulness of the SIC in preparing for internship and noted that the medium aided implementation.
A larger follow‐up study of 36 preinterns at three institutions was completed in spring of 2018, using the same preliminary SIC version, but additional item‐level analyses were performed to assess comfort with seven specific learning objectives. 13 After SIC implementation, a statistically significant improvement was detected in preintern comfort with identifying airway anatomy and physiology. Mean comfort increased, but was not statistically significant, for five of the six remaining objectives (determining necessity of diagnostic studies, interpreting ultrasounds, interpreting head computed tomography [CT] scans, constructing differential diagnoses, and describing basic biostatistics principles). One objective (interpreting radiographs) showed no change. Again, a majority of interns felt that Slack was an effective way to present the SIC.
Based on previous feedback, Slack provides an effective virtual classroom setting that fosters an interactive, collaborative environment for preintern learning. Because the SIC clinical questions are based on cases in the emergency department (ED), they help preinterns develop a clinical mindset in preparation for intern year, with the goal of bridging the relative educational gap after Match Day.
This year, we reconstructed the project, basing both the SIC and our assessments on the Milestones. Our objective was to incorporate a standardized, universally applicable set of learning criteria that could be used nationwide to address the relative educational gap between Match Day and the start of internship. The goal was to foster early, effective, collaborative learning within preintern cohorts as well as between preinterns and their future residency leaders, faculty, and coresidents. This curriculum was designed with the understanding that preinterns were concurrently juggling other responsibilities. Due to the nature of optional engagement, we acknowledged that variability in participation among the preintern cohort was a possible limitation to this study. Our hypothesis was that participation in the SIC would improve both CK and self‐reported perceived preparedness (PP) for residency.
METHODS
Study Design
This is a prospective observational study of the efficacy of a curricular invention and social media–based implementation.
Human Subject Committee Review
This study involved human subjects. Institutional review board (IRB) approval was obtained by the respective boards for each of the 11 collaborating institutions. The study was determined to be exempt by all institutions. Written informed consent was obtained via survey from all participating subjects.
Study Setting
The study's curricular implementation took place in a virtual classroom setting via the collaborative social media application Slack. This platform is a message board–based website on which private organizational workspaces are created. Real‐time messages, images, and files can be sent to the group, enabling rapid communication and collaboration. It can be accessed by the Internet or cellular data via computer, smartphone, or tablet. The designated SIC instructor at each collaborating institution created a workspace for their EM residency program. Within these workspaces, instructors had the capability of creating multiple channels. Every program had an Intern Cases channel on which to administer the SIC. After preinterns registered and joined their program's workspace, they were invited to the channel along with the designated instructor and (sometimes) other selected residents and faculty. There was no standardized practice for the number and professional role of instructors across institutions. (Programs had between one and four instructors. Instructors could be program directors, assistant program directors, clerkship directors, other faculty, or chief or senior residents.)
Study Population
The study population included newly matched EM preinterns who had not yet started clinical shifts at their new EM residency at our 11 collaborating institutions. 14 This did include preinterns who were either currently or previously in a preliminary or transitional year program. Also included were any preinterns who had graduated from medical school in a prior year. The SIC and postcurriculum survey were completed prior to their first clinical shift as an EM intern. Excluded were preinterns who had not participated at all in the SIC or had not participated in the 2019 Match (National Resident Matching Program). Also excluded were preinterns who did not indicate the institution for their new EM residency program.
Curricular Construction
We created the SIC using the American Board of Emergency Medicine Model of the Clinical Practice of EM as a framework. 15 Of the 23 existing Milestones, we selected the eight that could be addressed in a virtual classroom as the curricular focus. Milestones were selected based on teachability over Slack. Milestones that required face‐to‐face interaction, in‐person procedures, working in an actual ED, and demonstrating professionalism were deferred. We used all of the Level 1 Milestones because these represent the expectations of an incoming intern at the time they enter internship. Level 2 Milestones were also used because we wanted the capability to assess trends for preinterns who had already exceeded Level 1 Milestones (which is true in many cases; refer to Data Supplement S1, Appendix S1 [available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1111/acem.10503/full] for our selected Milestones and levels).
We set the total number of clinical cases at 10 to address our eight selected Milestones multiple times throughout the curriculum, yet still allow adequate time for curriculum completion between Match Day and the start of internship. We constructed common ED clinical scenarios for each case, incorporating a diverse spectrum of human body systems into the SIC (e.g., nervous system, cardiovascular, respiratory, orthopedics, gastroenterology, obstetrics, urogenital). We also addressed adult, pediatric, and geriatric topics (see Table 1 for more detail).
Table 1.
Curricular Design Matrix of Clinical Cases and Milestones
| Case | Milestone | Milestones Addressed/Case | |||||||
|---|---|---|---|---|---|---|---|---|---|
| #1 Stabilization | #3 DX Studies | #4 DDX | #5 Pharma | #7 Dispo | #9 Procedures | #10 Airway | #12 USS | ||
| STEMI | 1 | 1 | 1 | 1 | 1 | 5 | |||
| CVA | 1 | 1 | 1 | 1 | 1 | 5 | |||
| Trauma | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | |
| PE | 1 | 1 | 1 | 1 | 1 | 5 | |||
| SBO | 1 | 1 | 1 | 1 | 4 | ||||
| AAA | 1 | 1 | 1 | 1 | 1 | 1 | 6 | ||
| Ectopic | 1 | 1 | 1 | 1 | 4 | ||||
| APE | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 |
| Sepsis/Ger | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | |
| Asthma/Peds | 1 | 1 | 1 | 1 | 1 | 5 | |||
| 7 | 9 | 10 | 7 | 10 | 4 | 3 | 6 | ||
| Cases applicable to each milestone | |||||||||
Each case addresses multiple Milestones, and each Milestone is addressed by multiple cases, for maximum concept exposure and repetition of lessons across several types of clinical presentations.
AAA = abdominal aortic aneurysm; APE = acute pulmonary edema, CVA = cerebrovascular accident; Dispo = disposition; DDX = differential diagnosis; DX = diagnostic; Ger = geriatric; PE = pulmonary embolism; Peds = pediatrics; Pharma = pharmacotherapy; SBO = small bowel obstruction; STEMI = ST elevation myocardial infarction; USS = ultrasound.
We incorporated high‐resolution deidentified clinical images wherever possible in the SIC, including electrocardiograms, radiographs, ultrasounds, and CT images. These were taken from our own collections or from open source resources, such as Life in the Fast Lane (LITFL) and Academic Life in Emergency Medicine (ALiEM). The cases were each associated with five to nine intern‐appropriate clinical questions. We also constructed a case follow‐up with solutions to the discussion questions, clinical outcome, and links to appropriate supportive Free Open‐Access Medical Education (FOAMed) resources as well as references to traditional peer‐reviewed sources.
Kern's six steps of curriculum development served as our framework to identify gaps, conduct a needs assessment, develop goals and objectives, formulate an educational strategy, and plan the implementation and evaluation 16 (refer to Data Supplement S1, Appendix S2, for a sample case from the final product of our SIC).
Study Protocol
Each institution designated a faculty member or senior resident as the SIC instructor (refer to Data Supplement S1, Appendix S3, for complete setup and implementation instructions). The instructors each created their own Slack account and private workspace for their residency program. They then invited other EM program faculty and residents interested in medical education to this workspace.
Preinterns were invited via e‐mail to sign up for Slack and join their program's private workspace and intern cases channel. SIC instructors, preintern learners, and other invited faculty and residents were linked to their specific EM residency institution; via their private workspace, they were in communication with all who were involved with the SIC within their EM residency. Although SIC instructors had regular team communications with one another across all institutions via e‐mail, there was no established medium for preintern communication between institutions.
Once or twice a week, the instructors posted a new clinical case. Preinterns were given 48 to 72 hours to answer discussion questions and collaborate with each other on the message board. The instructors and other faculty or senior residents offered encouragement and clues whenever appropriate but did not reveal the answers. They were also available to answer follow‐up questions at the conclusion of each case or provide additional guidance as needed. Once the preinterns had ample time to answer the questions, the instructors posted the answers to the cases, as well as associated medical literature and media, including both FOAMed resources and traditional peer‐reviewed resources.
Instructors implemented the SIC between Match Day and the first clinical day of internship in 2019. Instructors typically took 15 to 20 minutes to set up the Slack workspace and intern cases channel. Each case required approximately 30 minutes to publish, provide encouragement, and provide the answers and resources. The instructors typically published one to two cases per week. Written subjective feedback was requested from the instructors and preinterns after course implementation to determine which aspects of the curriculum went well and which did not.
Key Outcome Measures
The pre‐ and postsurveys were purported to measure trends in preintern self‐reported PP. Measurement of PP was based on preintern levels of agreement with exact statements from the eight selected Milestones. The surveys were also purported to measure CK, based on preintern performance on untimed multiple choice examinations addressing topics from each of the selected Milestones. The surveys were only designed to measure changes for the eight Milestones taught with our virtual curriculum. All other Milestones were outside of our scope and were therefore excluded.
Funding
There were no funding organizations or sponsors for this study, and no such organization was involved in the conduct and reporting of this study.
Data Analysis
Sample size was determined by the number of consenting preinterns in the enrolled EM residency programs. To protect preintern confidentiality, no identifying information was collected, which limited our ability to complete paired statistical analysis. Thus, the Mann‐Whitney U‐test was used to compare pre‐ and postsurvey Likert scale (1–5) response distributions for PP. The Mann‐Whitney U‐test does not require paired results, it allows unequal numbers of pre‐ and postsurveys, and it assumes independence between the samples. Subjects completing the postsurvey were not given copies of their presurvey responses. Confidence interval (CI) assessment most commonly does not apply for this test. Student’s t‐tests compared CK examination scores between pre‐ and postsurvey groups; subgroup scores evaluated performance within the Milestones. Statistical analysis was conducted using the statistical programming language R, within the RStudio 3.6.1 shell.
RESULTS
Response Rate
Of the 151 preinterns contacted at our 11 collaborating institutions, 127 consented to enroll in the study (see Figure 1 for the enrollment flow chart). Separate participation calculations were completed for PP and CK survey sections. The attrition (lost‐to‐follow‐up) rates were calculated from the 127 total consenting participants.
Figure 1.
Preintern enrollment and retention flow sheet. CK=clinical knowledge; PP = perceived preparedness; F/U = follow‐up.
PP
Based on the Mann‐Whitney U‐test, statistically significant increases in PP median scores were found for 6 of the Milestone statements between pre‐ and postsurveys (Table 2). One additional Milestone statement (recognizing abnormal vital signs) demonstrated a nonstatistically significant increase in the PP median value.
Table 2.
Presurvey and Postsurvey PP (Likert Scale 1–5), Mann–Whitney U Summary Table
| Milestone Skill | Pretest Median | Posttest Median | p‐value | U |
|---|---|---|---|---|
| Recognizing abnormal vital signs | 4 | 5 | 0.320 | 5791 |
| Recognizing the unstable patient | 4 | 4 | 0.704 | 5549 |
| Ability to form a diagnostic plan | 4 | 4 | 0.699 | 5244 |
| Determining the need for diagnostic studies | 3 | 3 | 0.195 | 5922 |
| Ordering appropriate diagnostic studies | 3 | 3 | 0.994 | 5401 |
| Interpreting results of diagnostic studies | 3 | 3 | 0.657 | 5580 |
| Constructing a differential diagnosis | 4 | 4 | 0.160 | 4852 |
| Recognizing classes of medications and mechanisms of action | 3 | 3 | 0.280 | 4945 |
| Selecting appropriate medications | 3 | 3 | 0.0977 | 6080 |
| Recruiting appropriate clinical resources | 3 | 3 | 0.0757 | 6142 |
| Making admission or discharge decisions | 3 | 3 | 0.195 | 5939 |
| Assigning admitted patients to appropriate level of care | 3 | 3 | 0.0711 | 6153 |
| Describing pertinent anatomy for specific procedures | 3 | 3 | 0.136 | 6021 |
| Describing indications, contraindications, complications of ed procedures* | 3 | 3 | 0.00248 | 6661 |
| Describing upper airway anatomy* | 3 | 3 | 0.0327 | 6295 |
| Identifying procedure equipment and technique* | 2 | 3 | 0.00130 | 6746 |
| Identifying the pharmacology of RSI medications* | 2 | 3 | 0.00279 | 6655 |
| Ability to confirm placement of ETT* | 3 | 4 | 0.00415 | 6608 |
| Recognizing indications for ultrasound imaging | 4 | 4 | 0.258 | 5864 |
| Ability to optimize ultrasound images* | 2 | 3 | 0.0165 | 6418 |
| Interpretation of ultrasound images | 3 | 3 | 0.364 | 5783 |
ETT = endotracheal tube; PP = perceived preparedness; RSI = rapid sequence intubation.
Milestone topics for which preinterns reported statistically significant improvements in PP.
CK
Two‐sample t‐tests were conducted to compare overall examination scores between precurriculum and postcurriculum groups as well as subgroup scores to evaluate performance on Milestone‐specific questions (Table 3). There was no statistically significant difference in mean exam scores between presurvey (mean ± SD = 69.4% ± 0.090%) and postsurvey (mean ± SD = 69.8% ± 0.113%) groups (t(176) = −0.26, p = 0.793). There was a statistically significant improvement in performance for questions relating to Milestone 10 (airway management) between presurvey (mean ± SD = 2.278 ± 0.695) and postsurvey (mean ± SD = 2.524 ± 0.592; t(176) = −2.371, p = 0.0188 groups).
Table 3.
Clinical Knowledge Analysis Using Two‐sample Unpaired T‐tests for Pre‐ and Postsurvey Score Differences
| Measure | Pretest (±SD) | Posttest (±SD) | t‐value | p‐value |
|---|---|---|---|---|
| N | 115 | 63 | ||
| Overall mean examination scores | 69.4% (±0.090) | 69.8% (±0.113) | −0.2626 | 0.793 |
| Milestone 1 scores | 2.443 (±0.624) | 2.444 (±0.562) | −0.0102 | 0.992 |
| Milestone 3 scores | 1.809 (±0.907) | 1.841 (±0.971) | −0.2235 | 0.823 |
| Milestone 4 scores | 2.739 (±0.497) | 2.794 (±0.481) | −0.7079 | 0.480 |
| Milestone 5 scores | 12.487 (±2.566) | 12.127 (±2.691) | 0.8796 | 0.380 |
| Milestone 7 scores | 1.583 (±0.675) | 1.571 (±0.689) | 0.1049 | 0.917 |
| Milestone 9 scores | 1.922 (±0.796) | 2.016 (±0.813) | −0.7486 | 0.455 |
| Milestone 10 scores* | 2.278 (±0.695) | 2.524 (±0.592) | −2.3713 | 0.0188 |
| Milestone 12 scores | 11.539 (±3.193) | 11.698 (±3.114) | −0.3210 | 0.749 |
Milestone scores refer to the mean points obtained from questions testing those Milestones.
Statistically significant change.
DISCUSSION
After SIC implementation, statistically significant increases were seen in preintern PP for six of the Milestones and in CK for Milestone 10 (airway management). The Milestone statements that showed an increase in PP (such as airway management and goal‐directed focused ultrasound) may have been tasks to which preinterns had less exposure in medical school. This is in contrast to tasks more traditionally assigned to medical students (such as diagnostic study determination and diagnosis construction), for which preinterns reported high levels of PP on the presurvey.
The study was already highly relevant to medical educators at the time it was implemented in spring 2019, but we argue that it is even more relevant in 2020. The novel coronavirus is prevalent across the United States at the time of this article, limiting the ability for large gatherings in most of the country. Traditional methods of residency and medical student education, such as large‐group lectures and boot camps, need to be reimagined. In this study, we have shown based on preintern and instructor feedback that the SIC provides an alternative method for preinterns to learn, collaborate, and connect with each other and develop a community with their team prior to the start of residency. This is consistent with our previously published work.
Instructors and preinterns identified specific benefits to the SIC on surveys. Instructors appreciated the ease of use and clinical relevance of cases, and they stated that the SIC covered the basics they wanted their preinterns to know. They liked that it used multiple forms of media to review important EM practice topics. They found the cases to be a useful segue into conversations about their specific institutional practices, which helped preinterns learn what to expect. They enjoyed the ability to start teaching early, despite geographic and time zone separation, and they sensed excitement and enthusiasm from their preinterns. Consistent with prior years, they liked the engaging, round‐table style discussion, community building, and potential for collaboration between preinterns, senior residents, faculty, and residency leadership.
Preinterns appreciated that Slack offered a flexible, voluntary, and low‐pressure setting to refresh their knowledge before starting their internship. They liked the focus on bread‐and‐butter EM concepts, the step‐by‐step case walkthroughs, and the high‐quality clinical images and EM resources. They found the SIC to be an effective way to gauge their own knowledge deficits and stay on top of clinical medicine toward the end of the academic year. They also liked that the case questions also served as prompts for interesting discussions. Preinterns found it helpful to collaborate early with their own cohort, senior residents, and residency faculty and leadership and noted that it calmed their anxieties about starting their internship. They found that if unable to actively participate in a discussion in real time, they still benefited from reading through the previously completed discussions.
Instructors and preinterns also provided feedback on downsides to the SIC. The most common critique from instructors was insufficient preintern participation. Instructors felt conflicted about pushing preinterns to participate while vacationing or juggling other commitments. Instructors encouraged learner participation via e‐mail and Slack channel reminders. In an effort to increase preintern participation, some programs also recruited junior instructors (most commonly residents from varied training levels) to help moderate the cases. This strategy was implemented to add to the discussion and support team building between preinterns and their new senior residents. Supportive SIC instructor responses to preintern posts were also encouraged to enhance participation through positive reinforcement. Many instructors motivated preinterns to sign on to Slack by setting up an advice for interns channel wherein current residents could provide general guidance on moving, ending medical school, and starting internship. In addition to participation concerns, instructors noted that some of the cases were too simple and recommended adding more advanced EM concepts.
Preinterns agreed with instructors that variable participation was the biggest downside to the SIC. They noted the SIC was too long, and it was difficult to devote time to it while juggling other responsibilities such as moving homes. They felt it was somewhat burdensome to keep up with the case questions throughout the week, which were posted at unannounced times. Suggestions were made to spread out the case posts over a longer period of time (new cases once every 1–2 weeks, as opposed to one to two new cases per week). Additionally, suggestions were made to post the cases following a consistent, predetermined, and preannounced schedule.
Some preinterns expressed concern about posting incorrect answers and having their future program leaders read them, which felt like too much pressure. This effect may be heightened by the fact that mistakes posted on Slack remain permanently on the platform for all participants to see, which may deter participants from posting. Preinterns also were concerned that posting too often would make their cohort perceive them as too aggressive. For these reasons, answers were treated with encouragement (no differently than they would be in an in‐person classroom setting where there is naturally a wide spectrum of participation), and case discussions were never evaluated based on correctness. Anonymizing preinterns on Slack or randomizing them to other participating institutions would ameliorate these concerns, but doing so would also sacrifice the community building with coresidents and faculty because preinterns would not have the opportunity to collaborate with their future colleagues on the SIC material.
In terms of specific material addressed, preinterns requested more teaching on pharmacology, antibiotic selection, and ultrasound imaging in the SIC. They also noted some technical glitches such as difficulty setting up post notifications in Slack and variable Slack functionality displaying the clinical images, but a minority of participants noted these issues.
LIMITATIONS
There are limitations both to the research study and the curriculum itself. Regarding the study, PP and CK assessments only addressed the eight Milestones selected for inclusion in our curriculum, and thus our scope did not include a fully comprehensive ACGME Milestone‐based preintern assessment. Second, the CK assessment questions may have been inadequate to measure statistically significant changes. Sourcing CK topic questions from previously established and validated resources may ameliorate this issue, and if CK becomes our only outcome measure in subsequent studies, it would be reasonable to implement only SIC cases yielding statistically significant CK improvement. Also, there was no control group to compare trends in PP and CK between SIC intervention and control.
Importantly, the study was limited by variable learner participation in the SIC and surveys. Specifically, for PP there were 33% lost to follow‐up, and for CK there were 50% lost to follow‐up, attributed to both personal/extrinsic and Slack platform–related participation barriers as noted under Discussion. Bias may have been introduced because subjects who felt the most negatively or positively about the intervention may have been more likely to respond to the surveys than those who felt neutral.
Protecting the confidentiality of participants limited the ability to do a paired statistical analysis of pre‐ and postsurvey results. Additionally, because all data was aggregated for an extra layer of confidentiality protection, no further subgroup analysis could be performed for the institution‐specific effects of SIC implementation. Institution‐specific effects are expected to be relevant because of variables such as the professional roles of the designated SIC instructor and invited channel visitors (e.g., program directors versus senior residents). Subgroup analysis was not performed for the small number of preinterns who completed some residency training in a preliminary year, those who transitioned to EM after some training in another specialty, and others who had graduated from medical school in a previous year. It is likely there would have been an unequal comparison between these subgroups and the remainder of the preinterns. Our future work will address all these issues.
Limitations to the curriculum itself include the fact that it is designed only for use on Slack and has not been trialed for implementation in person, via recorded video, via live video conferencing, or via any other social media platform. Some residency institutions mandate the use of certain platforms (such as Microsoft Teams) for professional group communications and this may exclude these programs from the SIC. The curriculum uses only still images and no video clips, utilizing representative images for ultrasound and CT results. Future work on this project will address these curricular limitations.
Instructors and preinterns provided useful feedback on limitations they experienced during the research project. Instructors cited minor resolvable IRB issues as well as minor technical, Slack, and Internet access issues. They reported minor to moderate issues motivating preintern survey participation, which accounts for the aforementioned attrition rate. Instructors also perceived that preintern SIC participation was limited predominantly by vacation and family time; they noted that there were relatively few preinterns limited by medical school or preliminary year commitments or declining to participate altogether. Instructor perceptions were confirmed by the preintern breakdown of SIC participation barriers: in order of most to least commonly mentioned, these were vacation, family commitments, medical school/preliminary year commitments, research commitments, work commitments, and Slack access issues. Few reported no barriers at all.
CONCLUSIONS
After completing the Slack intern curriculum, EM preinterns reported significant improvements in perceived preparedness regarding several Milestone topics and showed improved clinical knowledge regarding Milestone 10 (airway management). This confirmed our perceived preparedness hypothesis, but additional studies are needed to evaluate changes in clinical knowledge. Graduate and undergraduate medical education leadership will benefit from the knowledge of preintern self‐reported perceived preparedness, as this information can augment medical training curricula and help fill knowledge gaps for preinterns based on their perceived strengths and weaknesses. Future directions include incorporation of validated, timed examinations to better assess trends in clinical knowledge. There also will be an investigation of the correlation between perceived preparedness and clinical knowledge and the preintern attributes that affect this correlation. Importantly, there will also be a well‐timed assessment of the utility of the Slack intern curriculum during the novel coronavirus pandemic, during which time medical students have commonly been displaced from clinical rotations altogether or have graduated early from medical school yet are not able to matriculate into EM residency until the standard dates in summer.
The authors are grateful to Drs. Angela Cirilli, Ellen Kurkowski, and Connie Yu for contributing high‐quality ultrasound images used in the 2019 Slack intern curriculum.
DISCLAIMER
Slack is a trademark of Slack Technologies Inc. The Slack Intern Curriculum research project is an independent research project and is not affiliated, sponsored, authorized, or otherwise associated with or by Slack Technologies Inc.
Supporting information
Data Supplement S1. Supplemental material.
AEM Education and Training 2021;5:1–10
Presented at the Society for Academic Emergency Medicine Annual Meeting, Denver, CO, May 2020, (accepted for presentation, conference canceled due to COVID‐19); American Academy of Emergency Medicine 26th Annual Scientific Assembly, Phoenix, AZ, April 2020 (accepted for top 6 oral abstract competition, conference canceled due to COVID‐19, and accepted for poster presentation, conference canceled due to COVID‐19); the Council of Emergency Medicine Residency Directors Academic Assembly, New York, NY, March 2020; and the Mediterranean Emergency Medicine Congress Abstract Competition, Dubrovnik, Croatia, September 2019 (awarded 6th place for oral abstracts).
The authors have no relevant financial information or potential conflict of interest to disclose.
Author contributions: MM—study concept and design; acquisition of the data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical expertise; and administrative, technical, or material support. TH—study concept and design, analysis and interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, and statistical expertise. AC—analysis and interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, and statistical expertise. AH—acquisition of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, and study supervision. JP—study concept and design, analysis and interpretation of the data, critical revision of the manuscript for important intellectual content, statistical expertise, and study supervision. JC and ES—drafting of the manuscript and critical revision of the manuscript for important intellectual content. MA, AC, GC, AC, LC, SG, MK, EL, DM, KN, AR, JR, KS, and RTS—acquisition of the data and critical revision of the manuscript for important intellectual content. MK—study concept and design; drafting of the manuscript; critical revision of the manuscript for important intellectual content; administrative, technical, or material support; and study supervision.
Supervising Editor: Teresa Chan, MD, MHPE.
References
- 1. Zeng W, Woodhouse J, Brunt LM. Do preclinical background and clerkship experiences impact skills performance in an accelerated internship preparation course for senior medical students? Surgery 2010;148:768–77. [DOI] [PubMed] [Google Scholar]
- 2. Jarou Z, Hillman E, Kellogg A, Lutfy‐Clayton L, Pelletier‐Bui A, Shandro J. Emergency Medicine Residents Association and Council of Residency Directors in Emergency Medicine Student Advising Guide. 2019. Available at: https://www.cordem.org/globalassets/files/student‐resources/2019‐studentadvisingguide‐final‐web‐version.pdf. Accessed on Nov 1, 2019
- 3. Franzen D, Kost A, Knight C. Mind the gap: the bumpy transition from medical school to residency. J Grad Med Educ 2015;7:678–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Lamba S, Wilson B, Natal B, Nagurka R, Anana M, Sule H. A suggested emergency medicine boot camp curriculum for medical students based on the mapping of Core Entrustable Professional Activities to Emergency Medicine Level 1 Milestones. Adv Med Educ Pract 2016;7:115–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Morgan H, Skinner B, Marzano D, Fitzgerald J, Curran D, Hammoud M. Improving the medical school‐residency transition. Clin Teach 2017;14:340–3. [DOI] [PubMed] [Google Scholar]
- 6. Minter RM, Amos KD, Bentz ML et al Transition to surgical residency: a multi‐institutional study of perceived intern preparedness and the effect of a formal residency preparatory course in the fourth year of medical school. Acad Med 2015;90:1116–24. [DOI] [PubMed] [Google Scholar]
- 7. Meier AH, Henry J, Marine R, Murray WB. Implementation of a Web‐ and simulation‐based curriculum to ease the transition from medical school to surgical internship. Am J Surg 2005;190:137–40. [DOI] [PubMed] [Google Scholar]
- 8. Clay AS, Andolsek K, Grochowski CO, Engle DL, Chudgar C. Using transitional year milestones to assess graduating medical students' skills during a capstone course. J Grad Med Educ 2015;7:658–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Beeson M, Christopher T, Heidt Jet alThe Emergency Medicine Milestones Project. A Joint Initiative of the Accreditation Council for Graduate Medical Education and the American Board of Emergency Medicine. Available at: https://www.acgme.org/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.pdf. Accessed May 20, 2019
- 10. Messman A, Malik A, Ehrman R. An asynchronous curriculum for teaching practical interpretation skills of clinical images to residents in emergency medicine. J Emerg Med 2020;58:299–304. [DOI] [PubMed] [Google Scholar]
- 11. Slack, Inc. Slack. 2020. Available at: https://slack.com/. Accessed Jun 15, 2020
- 12. Park JC, Kulkarni ML, McLean ME. The use of a social media based curriculum for newly matched interns transitioning into emergency medicine residency. JETem 2018;3:11–5. [Google Scholar]
- 13. Huls TA, McLean ME, Park JC, Anana MC, Klein MR, Kulkarni ML. Implementing a social media based curriculum for newly matched emergency medicine interns. West J Emerg Med 2019;20:S12–3. [Google Scholar]
- 14. Organization of Student Representatives, Association of American Medical Colleges, Handbook. 2016. Available at: https://www.aamc.org/system/files/2019‐08/gro‐osr‐handbook.pdf. Accessed on Nov 1, 2019
- 15. Counselman FL, Babu K, Edens MA et al The 2016 Model of the Clinical Practice of Emergency Medicine. J Emerg Med 2017;52:846–9. [DOI] [PubMed] [Google Scholar]
- 16. Kern DE, Thomas PA, Hughes MT, editors. Curriculum Development for Medical Education: A Six‐Step Approach. 2nd ed. Baltimore and London: Johns Hopkins University Press, 2009. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Supplement S1. Supplemental material.