Abstract
Study Objectives:
Sleep fellowship program websites likely serve as a preliminary source of information for prospective fellows. Arguably, applicants have likely become even more reliant on program websites during the COVID-19 pandemic due to travel restrictions and social-distancing measures limiting in-person interviews. In this study, we evaluated the content and comprehensiveness of sleep medicine fellowship websites to identify areas of improvement.
Methods:
A list of sleep medicine fellowship programs in the United States participating in the 2021 match cycle was compiled using the Electronic Residency Application Service (ERAS) and Fellowship and Residency Electronic Interactive database (FREIDA) websites. Twenty-two prespecified content criteria related to education, recruitment, and compensation were used to evaluate each program website. Sleep programs’ website comprehensiveness were compared based on US location, type, matching status, core specialty, and size of programs.
Results:
Seventy-eight US sleep fellowship program websites were evaluated. Most program websites had a working hyperlink on ERAS or FREIDA. There was considerable variability in content reported across program websites, with a mean of 56.8% of content items reported per program. There was a greater educational website content comprehensiveness for internal medicine compared with other specialty-based sleep programs. There was no difference in sleep programs’ website comprehensiveness based on US location, type, matching status, or size of programs.
Conclusions:
Website content comprehensiveness among sleep fellowship programs is variable. There is opportunity for all sleep fellowship programs to improve their websites to better inform prospective trainees.
Citation:
Shenoy S, Akberzie W, Landeo-Gutierrez JS, Leon Guerrero CR, Karroum EG. Evaluation of sleep medicine fellowship program websites. J Clin Sleep Med. 2023;19(6):1083–1088.
Keywords: sleep medicine, graduate medical education, website content, fellowship programs
BRIEF SUMMARY
Current Knowledge/Study Rationale: Fellowship programs’ websites are valuable tools for prospective applicants to learn about these programs, even more so during the COVID-19 pandemic. However, there is a paucity of data on the content of sleep fellowship programs’ websites. This study aims to investigate the comprehensiveness of the US sleep fellowship programs’ websites.
Study Impact: This study highlights the variability in website content comprehensiveness among sleep fellowship programs in the United States. There was a lower educational content on websites of noninternal, medicine-based sleep fellowship programs. It is hoped that reporting these results will help improve the website content of sleep programs and potentially allow prospective sleep fellows to make a better-informed decision when selecting and applying to these programs.
INTRODUCTION
Residency and fellowship program websites provide prospective fellows a preliminary glimpse into what a program has to offer.1–4 While this is typically followed by on-site interviews, tours, and meetings with faculty and current residents or fellows, information on a program’s website may heavily influence an applicant’s decision to apply to a program, and likely have an even greater influence on an applicant’s decision on selecting or ranking a program given the travel restrictions and social-distancing precautions since the start of the coronavirus disease 2019 (COVID-19) pandemic.
Indeed, multiple studies have demonstrated that residency and fellowship program websites are significantly utilized by applicants, but there is inconsistency and wide variation in content on these websites.1,2,5–13 In addition, multiple survey-focused studies have suggested that applicants base their decision not to apply to a program, or how to ultimately rank a program, on the information provided on a program’s website.1,2,4,14 Applicants have also noted that information on rotations and curriculum on a program’s website is most important to them.3,4,7 To date, there are no studies evaluating the comprehensiveness of sleep fellowship program websites. In this study, we sought to systematically evaluate the website content of sleep medicine fellowship programs in the United States in several domains pertinent to future applicants.
METHODS
Sleep medicine fellowship program identification
Sleep medicine fellowship programs in the United States were identified in the month of November 2020 using the Electronic Residency Application Service (ERAS) website15 and further verified with information from the Fellowship and Residency Electronic Interactive database (FREIDA) website.16 For context, the ERAS, overseen by the Association of American Medical Colleges (AAMC), is a centralized online application platform for residency and fellowship programs in the United States.15 The FREIDA is a searchable online database managed by the American Medical Association that catalogs information regarding residency and fellowship programs in the United States accredited by the Accreditation Council for Graduate Medical Education (ACGME).16 Of the sleep fellowship programs identified, only programs participating in the 2021 matching cycle of the National Resident Matching Program (NRMP)17 were included for our analysis. Sleep medicine fellowship program websites were accessed via hyperlinks on the ERAS and FREIDA websites. In cases where a hyperlink was nonfunctional or there was no listed web address, a Google search was utilized to locate the program website.
Evaluation of sleep medicine fellowship program websites
Three of the study authors who were sleep fellows in training at the time of this study (W.A., J.S.L.-G., and S.S.) independently evaluated the sleep programs’ websites between November 2020 and February 2021 using 22 prespecified content criteria (Table 1). The prespecified criteria were selected based on those used in a similar study5 and were modified and changed subsequently to include content criteria related to the COVID-19 pandemic as well as content specific and relevant to the sleep medicine field. Each website was independently reviewed by 2 reviewers. Discrepancies in data collection were arbitrated by consensus agreement between the 3 authors assessing the websites. The senior authors (C.R.L.G. and E.G.K) facilitated these discussions. Sleep medicine fellowship program websites were evaluated for the presence (yes/no) of each criterion (summarized in Table 1). Each website content criterion was listed as present if the information was identified directly on the sleep program website or was accessible from a link on the program website. The 22 prespecified criteria were further divided into the following categories: educational, recruitment, and compensation website content.
Table 1.
Content criteria used for evaluating the comprehensiveness of the sleep medicine fellowship programs’ websites (n = 78).
| Percentage of Programs Reporting Content Criteria | |
|---|---|
| Educational content criteria (n = 7) | |
| Telemedicine: Any mention of a sleep telemedicine clinic or rotation | 15.4% |
| Call schedule: At least 1 detail pertaining to call schedule | 16.7% |
| Advanced surgical therapies: Any mention of exposures to surgical subspecialities/procedures for sleep including ENT, maxillofacial surgery, and bariatric surgery | 44.9% |
| Active/past research: Specify ongoing or past projects, research topics, or scholarly activity that fellows and/or faculty have participated in | 74.4% |
| Rotation schedule: At least 1 rotation specified during the year of fellowship | 75.6% |
| Information on didactics: Any mention of a didactic topic (case presentation, journal club, or general didactics) | 80.8% |
| Rotation sites: Name of at least 1 clinical site for rotations | 92.3% |
| Recruitment content criteria (n = 12) | |
| COVID-19: Mention of changes made due to COVID-19 | 2.6% |
| Postgraduate placement: At least 1 alumnus listed with name and institution of post-fellowship employment | 29.5% |
| Message from the sleep fellowship program director: Welcome letter or message to applicants from the program director | 32.1% |
| Current fellows: At least 1 fellow listed with name and photo | 47.4% |
| Wellness: Webpage or paragraph titled “Wellness,” “Well-being,” or similar title including any mention of gym memberships, spousal support, or psychology/therapy resources | 50.0% |
| Visa information: Mention of visas a program does (or does not) support | 52.6% |
| Information about the area: At least 1 sentence about the location of the sleep fellowship program (ie, things to do, neighborhoods to live in) | 59.0% |
| Multidisciplinary faculty: Any mention of program having faculty from different specialties | 83.3% |
| Faculty information: At least 2 faculty members listed with name and subspecialty | 88.5% |
| Application requirements: At least 1 requirement to apply for fellowship | 89.7% |
| Contact information: Email address or phone number for the program director or the program coordinator | 98.7% |
| Program description: Any description of the sleep fellowship program | 98.7% |
| Compensation content criteria (n = 3) | |
| Housing: Subsidized housing options sponsored by sleep fellowship program or institution or mention that they offer a housing allowance | 11.5% |
| Benefits: At least 1 nonsalary component of compensation | 51.3% |
| Salary: Numerical values of salaries clearly printed | 53.8% |
COVID = coronavirus disease 2019, ENT = Ear, Nose, Throat.
Sleep medicine fellowship program stratification
Sleep medicine fellowship programs were stratified, and their website content comprehensiveness further analyzed, based on (1) US geographic location, (2) program type, (3) program 2021 matching status, (4) program core specialty, and (5) program size. Fellowship programs were grouped according to their US geographic location into 1 of 4 regions (Northeast, Midwest, South, and West) based on the US Census Bureau from 2010.18 Program type or affiliation (university- or community-based) was obtained from the program website. Program 2021 matching status (Complete Match: defined as all sleep positions filled; Incomplete Match: defined as either some or no positions filled) was obtained from the NRMP website.17 The sleep program core or related specialty (Internal Medicine or Other Specialties) for the academic year of 2020–2021 was obtained from the ACGME website.19 Program size was based on the number of available fellowship spots per class on the sleep fellowship program website.
Statistical analysis
Continuous variables were summarized by mean ± standard deviation, whereas the median and the range (minimum value–maximum value) were also reported. Categorical variables were summarized by percentage and frequency. Comparing 2 independent groups was performed using the nonparametric Mann–Whitney–Wilcoxon test for continuous variables, while comparing more than 2 independent groups was performed using the nonparametric Kruskal–Wallis test with post hoc analysis, using Dunn’s test when appropriate. The Spearman correlation coefficient (ρ) was used to analyze the association between 2 continuous variables. All statistical testing was 2-tailed and a P value below .05 was considered significant.
RESULTS
There were 78 different institutions with active sleep medicine fellowship training positions identified. Approximately two-thirds (62.8%) of the programs (49/78) had a functioning program website hyperlink on the FREIDA website. A higher number (74.4%) of programs (58/78) had a functioning program website hyperlink on the ERAS website. Most (80.8%) programs (63/78) had at least 1 functioning hyperlink from either the ERAS or FREIDA websites to their webpage, and 56.4% (44/78) of programs had a direct hyperlink to their webpage on both ERAS and FREIDA.
A total of 22 different content criteria were used to assess for comprehensiveness, including 7 items related to educational content, 12 items related to recruitment content, and 3 items related to compensation content (summarized in Table 1). The number of program websites that had each criterion varied greatly across the 22 content criteria studied, ranging from as low as 2 programs (2.6%) for “COVID-19” to as high as 77 programs (98.7%) for “Program description” and “Contact information.” More than half (59.1%) of the content criteria (13/22) were reported by less than two-thirds of programs, and 6 criteria were reported by fewer than one-third of the programs, including “Telemedicine,” “Call schedule,” “COVID-19,” “Postgraduate placement,” “Message from the Sleep Fellowship Program Director,” and “Housing.” The mean percentage of overall website comprehensiveness among sleep medicine fellowship programs was 56.8% ± 16.5% (median = 59.1%; range = 13.6–90.9%). The mean percentage of website comprehensiveness for education-related content was 57.1% ± 22.0% (median = 57.1%; range = 0–100%), for recruitment-related content was 61.0% ± 17.9% (median = 58.3%; range = 16.7–91.7%), and for compensation-related content was 38.9% ± 37.8% (median = 66.7%; range = 0–100%).
Analysis of website comprehensiveness by sleep medicine program characteristics
The overall, educational, recruitment, and compensation website content comprehensiveness by sleep medicine fellowship program characteristics is summarized in Table 2. There were no significant differences in the overall, educational, recruitment, and compensation comprehensiveness of programs across the 4 US regions. Likewise, the type of program (university- vs community-based) did not influence the overall, educational, recruitment, and compensation comprehensiveness of the sleep programs’ websites. In addition, the program’s matching status for 2021 (complete vs incomplete) did not impact the overall, educational, recruitment, and compensation comprehensiveness of their website content. Finally, with respect to the type of parent program, internal medicine–based programs had greater educational content comprehensiveness compared with other specialty-based programs (P = .002).
Table 2.
Sleep medicine fellowship program website comprehensiveness by characteristics.
| Category | No. of Programs | Comprehensiveness | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Overall | P | Educational | P | Recruitment | P | Compensation | P | ||
| US location | .147 | .837 | .106 | .386 | |||||
| Northeast | 20 | 61.4% ± 18.2% | 57.9% ± 23.4% | 67.5% ± 18.9% | 45.0% ± 43.6% | ||||
| Midwest | 21 | 58.4% ± 17.9% | 57.8% ± 26.1% | 62.7% ± 17.4% | 42.9% ± 35.2% | ||||
| South | 24 | 55.5% ± 14.2% | 57.1% ± 19.3% | 58.7% ± 16.6% | 38.9% ± 35.0% | ||||
| West | 13 | 49.3% ± 14.0% | 54.9% ± 19.2% | 52.6% ± 17.1% | 23.1% ± 37.0% | ||||
| Program type | .745 | .871 | .909 | .773 | |||||
| University | 71 | 56.8% ± 15.9% | 57.5% ± 21.0% | 60.9% ± 18.0% | 38.5% ± 37.6% | ||||
| Community | 7 | 56.5% ± 23.0% | 53.1% ± 31.6% | 61.9% ± 18.5% | 42.9% ± 41.8% | ||||
| Matching in 2021 | .408 | .572 | .082 | .964 | |||||
| Complete match | 66 | 56.1% ± 16.7% | 57.4% ± 22.6% | 59.7% ± 16.8% | 38.9% ± 38.6% | ||||
| No (or partial) match | 12 | 60.2% ± 15.8% | 56.0% ± 18.7% | 68.1% ± 22.4% | 38.9% ± 34.3% | ||||
| Related specialty | .088 | .002 | .909 | .461 | |||||
| Internal medicine | 57 | 58.9% ± 15.0% | 62.7% ± 17.0% | 61.3% ± 17.8% | 40.9% ± 36.7% | ||||
| Other specialty | 21 | 50.9% ± 19.3% | 42.2% ± 26.9% | 60.3% ± 18.6% | 33.3% ± 40.8% | ||||
Data are presented as mean ± standard deviation unless otherwise indicated. P values presented in bold are significant.
There was no significant correlation between the program size and the comprehensiveness of website content, including the overall website comprehensiveness (ρ = –0.030, P = .793), the educational content comprehensiveness (ρ = 0.143, P = .210), the recruitment content comprehensiveness (ρ = 0.008, P = .943), and the compensation content comprehensiveness (ρ = −0.143; P = .212).
DISCUSSION
We observed significant variability in the content included on US sleep medicine fellowship program websites. Previous studies examining other medical specialty program websites have also demonstrated significant variability in content included.5,8–13 Additionally, the low mean percentage of website comprehensiveness observed across sleep medicine programs was in line with prior studies in other medical fields.5,6,8–13
Notably, sleep programs are rarely stand-alone programs, and more often are a division or section within a larger parent teaching program. In subgroup analysis, the only significant difference we observed in website content across groups was between the type of parent program, with internal medicine–based programs having higher educational website comprehensiveness compared with non-internal medicine–based programs. One possible explanation for this could be related to the larger size and, in turn, likely greater resources of internal medicine programs in comparison to smaller parent programs like neurology or psychiatry. Indeed, prior studies have also demonstrated greater website comprehensiveness to be associated with larger programs.5,12 Another possible explanation is that internal medicine programs likely have a greater number of other associated fellowship programs, which may make them more apt, attuned, and experienced at maintaining websites.
We observed no correlation in the match rate of a program and the content comprehensiveness of a program website. Admittedly, the number of programs not completely filling was relatively small (n = 12) and may not be sufficient to detect a difference. However, this finding may suggest that applicants may find websites’ content valuable, but ultimately it has limited influence on their final selection of a program. Additionally, websites may aid in more intangible and qualitative aspects of a “successful match,” for both program and applicant that are immeasurable by match rate alone (eg, alignment with an applicant’s career goals, alignment with a program’s mission).
Given the timing of our analysis, it is important to highlight the impact of the COVID-19 pandemic on the recruiting and interview process. Indeed, prospective applicants greatly value the in-person interaction with residents, facilities, and surrounding area, as noted by a small pilot study conducted by the Department of Anesthesia at Loma Linda University.20 However, for most programs and for the safety of applicants during the pandemic, it has been imperative to conduct all interviews virtually. Unexpectedly and serendipitously, the pandemic heightened awareness surrounding inequity in the traditional in-person interview process. The virtual format eliminates travel and interview expenses that can disadvantage applicants with less financial means.21 Furthermore, the virtual interview process is arguably more time efficient for both the program and the applicant and also a more environmentally responsible approach.22 Consequently, the AAMC has recommended continuing the virtual interview process.23 With virtual interviews being the new norm, developing creative ways to better reach applicants, including videos and interactive content on websites, could be a way to enhance the virtual experience.
There are limitations to our study. There is not a widely accepted, standardized method of evaluating website content. However, many of the criteria used in this study have been used in similar previous studies and the selected criteria were agreed upon by the authors composed of sleep fellows and program directors. Additionally, the total number of programs is relatively small, limiting the power to detect smaller differences between various subgroups. Last, we only evaluated for the presence of content but did not assess for accuracy or quality. Conceivably, some websites likely have erroneous or outdated information. Also, there may be restrictions on what information is allowed and not allowed to be posted publicly, which cannot be accounted for in this study.
Overall, we believe that our findings in this study provide useful feedback for sleep medicine fellowship program directors and offer insight into how sleep programs can improve their websites to better attract prospective fellows. Additionally, improving a program’s website is a relatively easy way to create greater awareness of each teaching institution and to better inform prospective fellows.
ABBREVIATIONS
- COVID-19
coronavirus disease 2019
- ERAS
Electronic Residency Application Service
- FREIDA
Fellowship and Residency Electronic Interactive database
DISCLOSURE STATEMENT
All authors have seen and approved the manuscript. Work for this study was performed at the Department of Neurology & Rehabilitation Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC. The authors report no conflicts of interest.
REFERENCES
- 1. Chu LF, Young CA, Zamora AK, et al . Self-reported information needs of anesthesia residency applicants and analysis of applicant-related web sites resources at 131 United States training programs . Anesth Analg. 2011. ; 112 ( 2 ): 430 – 439 . [DOI] [PubMed] [Google Scholar]
- 2. Chen VW, Hoang D, Garner W . Do websites provide what applicants need? Plastic surgery residency program websites versus applicant self-reported needs . Plast Reconstr Surg Glob Open. 2018. ; 6 ( 10 ): e1900 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Mahler SA, Wagner MJ, Church A, Sokolosky M, Cline DM . Importance of residency program web sites to emergency medicine applicants . J Emerg Med. 2009. ; 36 ( 1 ): 83 – 88 . [DOI] [PubMed] [Google Scholar]
- 4. Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ . Aspects of residency programs’ web sites important to student applicants . Acad Emerg Med. 2005. ; 12 ( 1 ): 89 – 92 . [DOI] [PubMed] [Google Scholar]
- 5. Daniel D, Vila C, Leon Guerrero CR, Karroum EG . Evaluation of adult neurology residency program websites . Ann Neurol. 2021. ; 89 ( 4 ): 637 – 642 . [DOI] [PubMed] [Google Scholar]
- 6. Silvestre J, Tomlinson-Hansen S, Fosnot J, Taylor JA . Plastic surgery residency websites: a critical analysis of accessibility and content . Ann Plast Surg. 2014. ; 72 ( 3 ): 265 – 269 . [DOI] [PubMed] [Google Scholar]
- 7. Hashmi A, Policherla R, Campbell H, Khan FA, Schumaier A, Al-Mufarrej F . How informative are the plastic surgery residency websites to prospective applicants? J Surg Educ. 2017. ; 74 ( 1 ): 74 – 78 . [DOI] [PubMed] [Google Scholar]
- 8. Hansberry DR, Bornstein J, Agarwal N, McClure KE, Deshmukh SP, Long S . An assessment of radiology residency program websites . J Am Coll Radiol. 2018. ; 15 ( 4 ): 663 – 666 . [DOI] [PubMed] [Google Scholar]
- 9. Patel BG, Gallo K, Cherullo EE, Chow AK . Content analysis of ACGME accredited urology residency program webpages . Urology. 2020. ; 138 : 11 – 15 . [DOI] [PubMed] [Google Scholar]
- 10. Patel SJ, Abdullah MS, Yeh PC, Abdullah Z, Jayaram P . Content evaluation of physical medicine and rehabilitation residency websites . PM R. 2020. ; 12 ( 10 ): 1003 – 1008 . [DOI] [PubMed] [Google Scholar]
- 11. Miller VM, Padilla LA, Schuh A, et al . Evaluation of cardiothoracic surgery residency and fellowship program websites . J Surg Res. 2020. ; 246 : 200 – 206 . [DOI] [PubMed] [Google Scholar]
- 12. Skovrlj B, Silvestre J, Ibeh C, Abbatematteo JM, Mocco J . Neurosurgery residency websites: a critical evaluation . World Neurosurg. 2015. ; 84 ( 3 ): 727 – 733 . [DOI] [PubMed] [Google Scholar]
- 13. Huang BY, Hicks TD, Haidar GM, Pounds LL, Davies MG . An evaluation of the availability, accessibility, and quality of online content of vascular surgery training program websites for residency and fellowship applicants . J Vasc Surg. 2017. ; 66 ( 6 ): 1892 – 1901 . [DOI] [PubMed] [Google Scholar]
- 14. Embi PJ, Desai S, Cooney TG . Use and utility of Web-based residency program information: a survey of residency applicants . J Med Internet Res. 2003. ; 5 ( 3 ): e22 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Electronic Residency Application Service . ERAS participating specialties & programs. https://services.aamc.org/eras/erasstats/par/ . Accessed in November 2020.
- 16. American Medical Association . Fellowship and Residency Electronic Interactive Database (FREIDA). https://freida.ama-assn.org/ . Accessed in November 2020.
- 17. National Resident Matching Program (NRMP) . https://www.nrmp.org/match-data-analytics/fellowship-data-reports/ . Accessed in February 2021.
- 18. US Census Bureau . 2010 Census Regions and Divisions of the United States. https://www.census.gov/geographies/reference-maps/2010/geo/2010-census-regions-and-divisions-of-the-united-states.html . Accessed in November 2020.
- 19. Accreditation Council for Graduate Medical Education (ACGME) . Public. https://apps.acgme-i.org/ads/Public . Accessed in February 2021.
- 20. Bernstein SA, Gu A, Chretien KC, Gold JA . Graduate Medical Education virtual interviews and recruitment in the era of COVID-19 . J Grad Med Educ. 2020. ; 12 ( 5 ): 557 – 560 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Domingo A, Rdesinski RE, Stenson A, et al . Virtual residency interviews: applicant perceptions regarding virtual interview effectiveness, advantages, and barriers . J Grad Med Educ. 2022. ; 14 ( 2 ): 224 – 228 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Donahue LM, Morgan HK, Peterson WJ, Williams JA . The carbon footprint of residency interview travel . J Grad Med Educ. 2021. ; 13 ( 1 ): 89 – 94 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Association of American Medical Colleges . AAMC Interview Guidance for the 2022-2023 Residency Cycle. https://www.aamc.org/about-us/mission-areas/medical-education/aamc-interview-guidance-2022-2023-residency-cycle .