1. Introduction
Increasingly, applicants to various residencies and fellowships have utilized program websites for information. This trend has increased during the COVID-19 health crisis.1,2 Neurosurgical residents' interest in applying for functional neurosurgery fellowships are oftentimes reliant on online resources to identify programs of interest, as well as word of mouth. Functional neurosurgical fellowships do not participate in a centralized application or matching process, and applicants are required to research individual programs. This entails examining program requirements, applicant eligibility and the application process by using information that programs have provided online. In addition, a majority of programs often have only a single position. Providing applicants an adequate understanding of each programs’ nuance is paramount to ensure both applicant and program satisfaction (see Fig. 1, Fig. 2).
Notably, there are multiple previous studies reporting a paucity of available information for program websites across neurosurgery residency programs and various other specialties including fellowships.3, 4, 5, 6, 7 Importantly however, no critical analysis of functional neurosurgery program websites (FNW) has yet been published. The objective of our study was three-fold: 1) appraise available accessible information of available functional neurosurgical fellowship databases; 2) evaluate extent of education and recruitment information available on each FNW; and 3) assess associations between website comprehensiveness and program characteristics.
2. Materials and methods
2.1. Data sources and inclusion criteria
Two lists of functional neurosurgical fellowship programs are maintained by the American Society for Stereotactic and Functional Neurosurgery (ASSFN) and the American Association of Neurological Surgeons (AANS). Programs were excluded if they were not actively recruiting for a position in the 2021–2022. Although not a cross-sectional study, this study adheres to the guidelines for the Strengthening and Reporting of Observational Studies in Epidemiology as closely as possible. All available information was collected solely from program databases and websites.
2.2. Accessibility
Each functional neurosurgical fellowship database was evaluated based on the comprehensiveness of its program list, represented as the total number of advertised programs listed and accuracy of the list. The presence and functionality of the website links were evaluated for each program individually. Links were subsequently categorized as: direct and functional, indirect and functional (requiring multiple steps to access the program website), or non-existent.
2.3. Website analysis
Each FNW was evaluated by 2 independent reviewers (PG and LF) and corroborated by a third (RS) to limit bias or errors by any 1 reviewer. For each evaluated education and recruitment variable (described below), its presence or absence on each website was recorded, with no assessment of the quality of information provided. This allowed for a more objective and uniform evaluation of all websites and is based on methods used in similar studies.8,9
2.4. Website recruitment and education
Each FNW was analyzed for the availability of appropriate information used to recruit potential candidates and for the comprehensiveness of essential residency curriculum components. In total, 13 recruitment variables and 11 educational variables were selected based on the methodology of previous analyses of residency and fellowship website data (Table 1). The percentage of recruitment variables (n/13) and educational variables (n/11) found on each website was recorded. Descriptive statistics were subsequently calculated by category (see Table 2).
Table 1.
Recruitment | Education |
---|---|
Program description | Rotation schedule |
Email contact | On-call schedule |
Current fellow list | Didactic curriculum |
Salary information | Research opportunities |
Work hours | Research requirements |
Interview dates | Operative experience |
Graduate Information | Facility description |
Selection/evaluation criteria | Resident publications |
Social Life/Wellness initiatives | Faculty list available |
Domestic considerations | Fellowship publication list available |
Program size information | Fellowship conference presentations available |
Meal allowance information | |
Debt management information |
Table 2.
Characteristic | No. (%) of programs (n = 35) | Mean (SD) education content in FNWs, % | Mean (SD) recruitment content in FNWs, % |
---|---|---|---|
Region | p = 0.56 | p = 0.06 | |
South | 8 (22.9) | 46.15 (9.1) | 34.62 (8.4) |
Northeast | 6 (17.1) | 35.9 (7.9) | 23.08 (6.1) |
Midwest | 7 (20.0) | 28.6 (8.8) | 25.27 (9.6) |
West | 8 (22.9) | 44.44 (10.2) | 34.6 (8.4) |
Outside U.S. | 6 (17.1) | 28.1 (7.8) | 19.23 (6.8) |
Number of Fellows | p = 0.84 | p = 0.90 | |
1 | 29 (82.9) | 38.2 (9) | 27.8 (7.3) |
2 | 5 (14.3) | 41.0 (10) | 29.5 (9.3) |
Number of faculty | p = 0.30 | p = 0.09 | |
1-15 | 21 (60.0) | 42.14 (9.6) | 40 (8.2) |
≧16 | 14 (40.0) | 32.05 (8.5) | 18.1 (7.1) |
Ranked hospital | p = 0.90 | p = 0.90 | |
Yes | 13 (37.1) | 36.5 (9.9) | 29 (8.7) |
No | 22 (62.9) | 38.8 (9) | 27.6 (7) |
Abbreviation: FNWs, functional neurosurgery websites.
2.5. Program comparison
Functional neurosurgery fellowship programs were categorized by characteristics such as geographic location, program size, and ranking. Programs were grouped geographically by the region of the US in which they are located (South, West, Midwest, or Northeast) or international. Program size was classified by the number of available functional neurosurgical fellowship positions and the number of faculty neurosurgeons. Faculty size was separated into programs with 1–15 vs 16 or more faculty. Program ranking was determined by the listing of the 2022 U S. News & World Report.
2.6. Statistical analysis
The mean (SD) percentages of education and recruitment content were compared among groups with analysis of variance t tests, or ANOVA as required for the number of variables in comparison groups. Post-hoc pairwise differences and confidence intervals were constructed using the Tukey method where applicable. All hypothesis tests were 2-sided, with p < 0.05 used as the threshold for statistical significance. The statistical analysis was performed using GraphPad Prism version 8.0.0 for Windows (GraphPad Software, USA).
3. Results
3.1. Accessibility of information
The AASFN website listed 43 functional neurosurgical programs, and the AANS website included 38 functional neurosurgical programs. Following identification of the listed programs, contact information and website review for each individual program was utilized to identify if an available neurosurgical fellowship program existed at the institution. In total, 35 programs that were planning to accept applications for the 2021–2022 were included for analysis (Supplemental Material).
Of the 35 programs, the AASFN database provided website links for 32 (81.4%) of programs, with email contacts in 27 (77.1%) and phone number contacts in 31 (88.6%) (Table 3). Of the available website links, only 10 (28.6%) directly linked to the FNW, while 9 (25.7%) were non-functional, and 16 (45.7%) required multiple steps before FNW access. An alternative source, the AANS database, provided email contacts to 28 (80.0%) programs, and phone contacts to 31 (88.6%) programs. Notably, the AANS did not provide hyperlinks to FNWs. Of the 35 programs accepting applications, FNW were accessible via Google query in 30 (85.7%) programs.
Table 3.
Characteristic | ASSFN | AANS |
---|---|---|
Neurosurgery residencies listed | 43 | 38 |
Fellowships accepting applications | 35 (81.4) | 35 (92.1) |
Phone Number Contact Available | 31 (88.6) | 31 (88.6) |
Email Contact Available | 27 (77.1) | 28 (80.0) |
Website link Available | 31 (88.6) | 0 |
Non-functioning links | 9 (25.7) | NA |
Links requiring multiple steps | 16 (45.7) | NA |
Direct links | 10 (28.6) | NA |
Website link Available via Google Query | 30 (85.7) |
Abbreviations: ASSFN, American Society for Stereotactic and Functional Neurosurgery, AANS, American Association of Neurological Surgeons; NA, not applicable.
a Values are No. Or No (%).
b AANS did not list whether programs were accepting applications.
3.2. Fellowship recruitment
Thirteen different features were evaluated within the recruitment aspect of individual FNWs (Table 4). Of the 35 FNWs, programs displayed an average of 3.66 (28.1%) of the 13 available features. The most frequently listed features in this category were program contact email (82.9%), program description (74.3%), recruitment elevation/selection criteria (51.4%), wellbeing/social information (34.3%), and city information (28.6%) in descending order. Work hours and meal allowance were seldom included (2.9%), and debt management information was not included on any program websites.
Table 4.
Fellowship | Recruitment Subscore | Education Subscore | Cumulative Score | US News Ranking |
---|---|---|---|---|
Vanderbilt University Medical Center | 7 | 9 | 16 | 48 |
University of Utah | 8 | 8 | 16 | Unranked |
University of Florida | 6 | 9 | 15 | Unranked |
West Virginia University School of Medicine | 8 | 7 | 15 | Unranked |
Thomas Jefferson University | 7 | 7 | 14 | Unranked |
Baylor College of Medicine | 6 | 7 | 13 | 33 |
University of Colorado | 6 | 7 | 13 | Unranked |
Emory University | 5 | 8 | 13 | 22 |
University of Toronto/Toronto Western Hospital | 5 | 8 | 13 | Unranked |
Allegheny General Hospital | 6 | 6 | 12 | Unranked |
University of British Columbia | 6 | 6 | 12 | Unranked |
University of California, San Francisco | 7 | 5 | 12 | 1 |
Cleveland Clinic Foundation | 6 | 6 | 12 | 7 |
Barrow Neurological Institute/St. Joseph's Hospital | 4 | 7 | 11 | 27 |
Johns Hopkins | 3 | 7 | 10 | 4 |
Ohio State University | 4 | 6 | 10 | 28 |
Oregon Health Sciences University | 4 | 6 | 10 | Unranked |
University of Pittsburgh School of Medicine | 3 | 7 | 10 | Unranked |
Swedish Medical Center | 3 | 6 | 9 | Unranked |
Walton Centre for Neurology & Neurosurgery | 3 | 6 | 9 | Unranked |
University of Alabama | 2 | 6 | 8 | Unranked |
University of California, Los Angeles | 2 | 6 | 8 | 8 |
Virginia Commonwealth University | 2 | 6 | 8 | Unranked |
University of Nebraska Medical Center | 4 | 3 | 7 | Unranked |
University of California, Irvine | 2 | 3 | 5 | Unranked |
University of Illinois at Chicago | 3 | 2 | 5 | Unranked |
Penn State Hershey Medical Center | 0 | 5 | 5 | Unranked |
Case Western Reserve University | 2 | 1 | 3 | 39 |
Yale University School of Medicine | 2 | 1 | 3 | 42 |
Rush University Medical Center | 1 | 1 | 2 | 3 |
University of Western Ontario | 1 | 1 | 2 | Unranked |
University of Louisville | 0 | 1 | 1 | Unranked |
Mount Sinai Medical Center | 0 | 1 | 1 | 10 |
Triservice Hospital | 0 | 1 | 1 | Unranked |
University of Calgary | 0 | 0 | 0 | Unranked |
3.3. Fellowship education
A total of 11 different features were evaluated within the education aspect of individual FNWs. Of the 35 FNWs, programs displayed an average of 5.94 (45.7%) of the 11 available features. The most frequently listed features in this category were comprehensive faculty list (97.1%), facility description (71.4%), research opportunities (71.4%), operative experiences (60.0%), and education curriculum/didactic session information (60.0%). Call schedule information (5.7%), fellowship conferences/funding (5.7%), and rotation schedule information (8.6%) were seldom included on program websites.
3.4. Program comparison
Individual functional neurosurgical fellowship programs were stratified by program characteristics and compared for FNW comprehensiveness across the individual domains of fellowship recruitment and education.
Region within the United States or outside was not associated with greater FNW education (p = 0.56) or recruitment (p = 0.06) content. Program size, approximated by grouping the number of faculty members (n = 1–15, or n = 16+) was not associated with greater FNW education (p = 0.30) or recruitment (p = 0.09) comprehensiveness. Program institution being ranked or unranked did not correlate with FNW comprehensiveness across education (p = 0.90) or recruitment (p = 0.90). The number of fellows at a respective program was not significantly associated with recruitment content (p = 0.90) or education (p = 0.56) content on individual FNWs.
3.5. Comparison with other specialty fellowship programs
Although there is limited available data of website analyses within neurosurgical fellowship programs, previous evaluation has been completed for fellowships in other selected disciplines. Available results of comparable analyses on different fellowship programs using available criteria are presented descriptively in Table 5.
Table 5.
Study Information |
% of Programs |
||||||||
---|---|---|---|---|---|---|---|---|---|
Author, Year | Surgical Discipline | Fellowship Specialty | Number of Programs | Number of Websites | Description (Rank) | Faculty List (Rank) | Fellow List (Rank) | Research Opportunities (Rank) | Operative Case Listing (Rank) |
Silvestre, 201614 | Plastic Surgery | Craniofacial Surgery | 28 | 24 | 95.8 (4) | 83.8 (3) | 41.7 (2) | 70.8 (3) | 75.0 (3) |
Huang, 201713 | General Surgery | Vascular Surgery | 94 | 89 | 97.8 (3) | 89.9 (2) | 10.1 (7) | 34.8 (7) | 21.3 (6) |
Shaath, 201819 | Orthopedic Surgery | Trauma Surgery |
54 | 51 | 100 (1) | 29 (6) | 35 (4) | 45 (5) | 76 (2) |
Maisner, 202112 | Plastic Surgery | Aesthetic Surgery | 28 | 28 | 85.7 (6) | 17.9 (7) | 32.1 (5) | 35.7 (6) | 35.7 (5) |
Gerlach, 202116 | Orthopedic-/Neuro-Surgery | Spine Surgery | 74 | 74 | 87.8 (5) | 77.0 (4) | 43.2 (1) | 81.1 (1) | 78.4 (1) |
Aryanpour, 202215 | General Surgery | Surgical Oncology | 88 | 88 | 100 (1) | 64.0 (5) | 40.0 (3) | 56.0 (4) | 12.0 (7) |
Present Study | Neurosurgery | Functional Neurosurgery | 43 | 35 | 81.4 (7) | 97.1 (1) | 25.7 (6) | 71.4 (2) | 60.0 (4) |
4. Discussion
The use of online resources such as websites for recruitment information for fellowship applicants has been increasingly utilized due to clear benefits of ease-of-use, low-cost, and the ability to reach international viewership. Even though websites for residency and fellowship programs have existed for decades, there continues to be a clear paucity of information useful to applicants as well as considerable heterogeneity in the provided information. Previous analyses of neurosurgical program websites have been limited to residency programs, and to date, no critical analysis of website information for available neurosurgical fellowships has been completed. Given the smaller comparative number of functional neurosurgical fellowship programs compared to other fellowships, it is paramount that available resources reflect programs with accuracy and provide applicants with necessary information to ensure both applicant and program success.
Within our study, the availability of recruitment features such as program email contacts descriptions, and evaluation criteria were often reported; however there was a significant paucity of available information outside of these criteria as less than half of programs have any additional information. This indicates a significant barrier to the applicant, since it requires the additional steps of sending follow-up emails or calling program coordinators to identify further information about an individual program. Importantly, this could reduce an applicant's interest in a program that would otherwise represent a great “fit”, resulting in suboptimal recruitment.6,9
Of the education information identified on FNWs in our study, factors such as faculty list, facility description, operative experience, curriculum, and research opportunities were most often included. This is in keeping with neurosurgical residency program websites as well as fellowship programs in non-neurosurgical disciplines such as radiology, which have identified research opportunities and output in 40–90% of program websites.4,6,8,9 Neurosurgical academic productivity has considerably increased over time, as demonstrated in bibliometric analyses inclusive of all neurosurgical sub-disciplines.10 Functional neurosurgery has seen a number of research advances. It continues to be an exciting field on the forefront of innovation. Correspondingly, it is well-purposed to outline available research opportunities within individual FNWs.11
Outside of neurosurgery, a number of surgical fields including general surgery, plastic surgery, and ophthalmology have demonstrated significant challenges with access to comprehensive information for their respective fellowships.12, 13, 14, 15, 16 For example, in the case of spine surgery – a fellowship of both orthopedic surgery and neurosurgery – there is significant variation in the type of information provided by the North American Spine Society (NASS) database and individual program websites.16 Specifically, their results indicated that the NASS database highlighted general information about individual programs more consistently, while individual websites provided more complete information on fellow roles and responsibilities. Their results are consistent with our findings in this study, which demonstrated the utility of the AASFN and AANS databases as initial resources, especially when utilized in conjunction with individual FNWs. Importantly, critical analyses of other surgical fellowship websites have revealed a lack of consolidation and consistency in the online content provided by individual programs.14, 15, 16
Previous investigations have highlighted that program websites are an important factor in both residency and fellowship application processes. In a study of 188 applicants to emergency medicine, 78% indicated that information on program websites influenced their decision to apply to a program with 41% indicating they had decided not to apply to a program based upon the quality of the program's respective website.17 Although no such study has been replicated in applicants to functional neurosurgery, this demonstrates that applicants place a significant weight on program websites in their decision process. Due to the COVID-19 pandemic, many applicants to residency programs indicated that program social media activity also had a significant effect on their decision to apply to a program.18 Together, this positions a program's online information as paramount in an applicant's decision. Applicants to fellowship programs such as functional neurosurgery may span international borders. This places an even greater onus on gathered online information since traveling to programs can cause significant financial burden on applicants.
To our knowledge, our study is the first to provide critical analysis of the websites of functional neurosurgical fellowship programs. This study does exist with limitations. First, despite conducting an inclusive search, it is possible that certain information may have been missed due to variation in program websites. Second, although we attempted to limit this through the use of multiple reviewers, evaluator bias may have affected conversion of available website data to binary variations. Finally, despite our efforts to identify programs from the available ASSFN and AANS databases, it is possible that some fellowship programs may not be indexed or available through search engine query, and may have been missed.
5. Conclusion
Although many functional neurosurgery programs have maintained comprehensive FNWs, there are clear deficiencies in important aspects of both available recruitment and education content. These results indicate an opportunity for the functional neurosurgery community to expand the breadth and organization of available program information to ensure that these databases and individual websites are comprehensive. This will benefit programs and applicants by enabling the best mutual fit.
Credit author statement
Peter Gariscsak: Writing – original draft, Investigation. Luisa Figueredo: Writing – Investigation. Rohin Singh: Investigation, Conceptualization. Henry O. Stonnington: Review & Editing. Nolan Brown: Conceptualization. Shane Shahrestani: Review & Editing. India C. Rangel: Review & Editing. Matthew T. Neal: Review & Editing, Supervision. Naresh P. Patel: Review & Editing, Supervision.
Declaration of competing interest
The authors have no conflicts of interest to declare.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.wnsx.2023.100184.
Appendix A. Supplementary data
The following is the Supplementary data to this article.
References
- 1.Tong J.Y., Hartkorn A.R., Sataloff R. Otolaryngology residency program websites in a socially distanced application cycle. Ear Nose Throat J. 2022 doi: 10.1177/01455613211064005. Published online January 5. [DOI] [PubMed] [Google Scholar]
- 2.Tang O.Y., Ruddell J.H., Hilliard R.W., Schiffman F.J., Daniels A.H. Improving the online presence of residency programs to ameliorate COVID-19's impact on residency applications. Postgrad Med. 2021;133(4):404–408. doi: 10.1080/00325481.2021.1874195. [DOI] [PubMed] [Google Scholar]
- 3.Mukhdomi T., Woolley P., Mukhdomi J., et al. An evaluation of the online quality of content of United States accredited pain medicine fellowship training programs. Anesthesiol Pain Med. 2021;11(3) doi: 10.5812/aapm.115981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Skovrlj B., Silvestre J., Ibeh C., Abbatematteo J.M., Mocco J. Neurosurgery residency websites: a critical evaluation. World Neurosurg. 2015;84(3):727–733. doi: 10.1016/j.wneu.2015.04.051. [DOI] [PubMed] [Google Scholar]
- 5.Ruddell J.H., Hartley-Blossom Z.J., Bajaj A.I., Grand D., Eltorai A.E.M. Analysis of Abdominal Radiology fellowship website content and comprehensiveness. Abdom Radiol. 2019;44(4):1601–1605. doi: 10.1007/s00261-018-1861-0. [DOI] [PubMed] [Google Scholar]
- 6.Hsu A.L., Chien J.L., Sarkany D., Smith C. Evaluating neuroradiology fellowship program websites: a critical analysis of all 84 programs in the United States. Curr Probl Diagn Radiol. 2021;50(2):147–150. doi: 10.1067/j.cpradiol.2019.11.002. [DOI] [PubMed] [Google Scholar]
- 7.Lee D.J., Jung J., Daniels A.H., Eltorai A.E.M. Analysis of musculoskeletal radiology fellowship websites. Skeletal Radiol. 2020;49(7):1149–1153. doi: 10.1007/s00256-020-03401-3. [DOI] [PubMed] [Google Scholar]
- 8.Singh R., De La Peña N.M., Azuma A.F., Smaga B.W., Pollock J.R., Patel N.P. Letter to the editor: analysis of neurosurgery residency websites by educational and recruitment information in 2020. World Neurosurg. 2021;151:307–308. doi: 10.1016/j.wneu.2021.04.140. [DOI] [PubMed] [Google Scholar]
- 9.Vilanilam G.K., Wadhwa V., Purushothaman R., Desai S., Kamran M., Radvany M.G. Critical evaluation of interventional neuroradiology fellowship program websites in North America. World Neurosurg. 2021;146:e48–e52. doi: 10.1016/j.wneu.2020.09.164. [DOI] [PubMed] [Google Scholar]
- 10.Hauptman J.S., Chow D.S., Martin N.A., Itagaki M.W. Research productivity in neurosurgery: trends in globalization, scientific focus, and funding. J Neurosurg. 2011;115(6):1262–1272. doi: 10.3171/2011.8.JNS11857. [DOI] [PubMed] [Google Scholar]
- 11.Artusi C.A., Ramirez-Zamora A., Bozzali M. Editorial: advances in functional neurosurgery. Front Neurol. 2021;12 doi: 10.3389/fneur.2021.812100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Maisner R.S., Babu A., Ayyala H.S., Ramanadham S. How comprehensive are aesthetic surgery fellowship websites? Ann Plast Surg. 2021;86(6):615–617. doi: 10.1097/SAP.0000000000002518. [DOI] [PubMed] [Google Scholar]
- 13.Huang B.Y., Hicks T.D., Haidar G.M., Pounds L.L., Davies M.G. An evaluation of the availability, accessibility, and quality of online content of vascular surgery training program websites for residency and fellowship applicants. J Vasc Surg. 2017;66(6):1892–1901. doi: 10.1016/j.jvs.2017.08.064. [DOI] [PubMed] [Google Scholar]
- 14.Silvestre J., Agarwal D., Taylor J.A. Craniofacial surgery fellowship websites. J Craniofac Surg. 2016;27(4):831–834. doi: 10.1097/SCS.0000000000002598. [DOI] [PubMed] [Google Scholar]
- 15.Aryanpour Z., Ananthasekar S., Rajan S.S., Reddy S. Evaluation of surgical oncology fellowship websites: are we showing what applicants need to see? Surg Open Sci. 2022;7:1–5. doi: 10.1016/j.sopen.2021.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Gerlach E.B., Plantz M.A., Swiatek P.R., et al. The content and accessibility of spine surgery fellowship websites and the North American Spine Surgery (NASS) fellowship directory. Spine J. 2021;21(9):1542–1548. doi: 10.1016/j.spinee.2021.04.011. [DOI] [PubMed] [Google Scholar]
- 17.Gaeta T.J., Birkhahn R.H., Lamont D., Banga N., Bove J.J. Aspects of residency programs' web sites important to student applicants. Acad Emerg Med. 2005;12(1):89–92. doi: 10.1197/j.aem.2004.08.047. [DOI] [PubMed] [Google Scholar]
- 18.Naaseh A., Thompson S., Tohmasi S., et al. Evaluating applicant perceptions of the impact of social media on the 2020-2021 residency application cycle occurring during the COVID-19 pandemic: survey study. JMIR Med Educ. 2021;7(4) doi: 10.2196/29486. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Shaath M.K., Yeranosian M.G., Ippolito J.A., Adams M.R., Sirkin M.S., Reilly M.C. Evaluation of the content and accessibility of web sites for accredited orthopaedic trauma surgery fellowships. J Bone Jt Surg. 2018;100(9) doi: 10.2106/JBJS.17.01112. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.