Abstract
Background:
As orthopaedic surgery remains a highly competitive specialty, YouTube has emerged as a major online resource for medical students seeking guidance for residency applications. The credibility, thoroughness, and representation of the advice provided in these videos warrant a critical analysis.
Methods:
A YouTube search was conducted, and the top 100 videos were screened. Seventeen of the videos met the evaluation criteria. Three authors independently assessed these videos for 23 significant residency application variables. Content creators' qualifications, viewership, sex, and racial representation were also evaluated. Discrepancies were resolved through joint review and consensus.
Results:
Of the 17 evaluated YouTube videos on orthopaedic surgery residency, research experience and the United States Medical Licensing Exam Step 1 score were the most discussed variables. Videos hosted by orthopaedic physicians received fewer views on average than those hosted by nonorthopaedics. Minority representation varied, with Asian-identifying creators receiving the highest average views. Male-hosted videos had greater viewership compared with female creators.
Conclusion:
YouTube videos on orthopaedic surgery residency focused on research experience, reflecting changes in National Resident Matching Program's application evaluation metrics. The ambiguity of advice on research type and underemphasis on other crucial factors, such as letters of recommendation and interview performance, suggest the need for more comprehensive guidance. Moreover, the videos' demographic disparity compared with the actual field indicates the need for more diverse representations among content creators. We recommend that orthopaedic organizations create tailored and comprehensive guidance for prospective applicants.
Introduction
Orthopaedic surgery has long been a competitive specialty in medicine, consistently ranking among the top 5 most competitive specialties in the United States.1 According to the 2023 National Resident Matching Program (NRMP) data, only 72.9% of US allopathic (MD) and 50.2% of osteopathic (DO) graduate applicants successfully matched into orthopaedic surgery, underscoring the specialty's competitiveness.1
YouTube’s role in providing advice to medical students interested in orthopaedic surgery residency remains unexplored. With the increasing importance of YouTube as a platform for guidance, a critical question arises: What is the thoroughness and credibility of advice given in the most viewed YouTube videos aimed at guiding medical students in their application to orthopaedic surgery residencies and do these videos represent various demographics that are present in the field? To answer this question, our study aimed to critically analyze the top-viewed YouTube videos that offer advice to applicants in matching orthopaedic surgery residency. Comprehensive evidence-based advice from experienced professionals should encompass multiple key factors that reflect a holistic approach to the residency process.
Recognizing the need for such comprehensive advice on YouTube videos, the authors undertook an in-depth examination of this media content. Videos were selected based on passing an exclusion criterion and evaluated for the discussion of numerous variables that residency program directors (PDs) and current literature deem important, including but not limited to the United States Medical Licensing Exam (USMLE) scores, clerkship grades, and research experience1-7. The speaker's credibility, based on education and profession, was also analyzed to assess the advice provided. The authors also noted speaker self-identified demographics in the selected videos to assess the representation of various gender identities and ethnicities.
Methods
On April 20, 2023, the authors conducted a search on YouTube using the phrase, "Advice on matching into Orthopedic Surgery". The authors used YouTube's “relevant” filter, which prioritizes videos based on factors such as keyword matching, popularity, and upload date. This is the only filter setting that YouTube contains to yield the most relevant videos regarding a user search. YouTube did not disclose the total number of videos yielded by the search when it was based on “relevant” filter settings. For this reason, the authors screened the top 100 videos that populated the search based on the following exclusion criteria: videos that were not in English, videos unrelated to orthopaedic surgery residency application information, and duplicate videos.
Following this exclusion criterion, 17 videos were used for the evaluation. Based on recent NRMP data and a relevant literature review on orthopaedic surgery residency applications, the authors identified 23 variables that consistently influence the matching process1-7. These variables can be broadly classified into categories, such as academic achievement (e.g., USMLE Step 1 score, clerkship grades, preclerkship grades), professional experiences (e.g., research experiences, work experiences, volunteer experiences), mentoring and networking (e.g., orthopaedic mentors, away rotations), application components (e.g., letters of recommendation and personal statements), and personal qualities (e.g., communication and reliability). In addition, specific circumstances related to applicants (e.g., MD and DO degree holders, US and International Medical Graduate [IMG] applicants), shifts in policy (e.g., Step 1 becoming pass/fail [P/F], shifting to a holistic approach), and decision-making aspects (e.g., choosing a residency program) were considered1,4,5. The complete list of the variables is presented in Table I.
TABLE I.
Analyzed Orthopaedic Residency Advice Variables on YouTube*
| Variable | Videos n = 17 (%) |
|---|---|
| Research experience | 15 (88.23) |
| USMLE STEP 1 score | 15 (88.23) |
| Applicant personal attributes | 13 (76.47) |
| Research years | 11 (64.71) |
| Specialty attributes | 10 (58.82) |
| USMLE STEP 2 score | 10 (58.82) |
| Letters of recommendation | 9 (52.94) |
| Importance of interviews | 9 (52.94) |
| Orthopaedic mentors | 8 (47.06) |
| Away rotations | 8 (47.06) |
| United States vs. international medical graduates | 6 (35.29) |
| Work experience | 5 (29.41) |
| Clerkship grades | 5 (29.41) |
| Class rank/Alpha Omega Alpha status | 5 (29.41) |
| Personal statement | 5 (29.41) |
| Choosing a program | 4 (23.53) |
| Volunteer experience | 4 (23.53) |
| Medical school institution | 4 (23.53) |
| Step 1 becoming pass/fail | 4 (23.53) |
| National resident matching program reference | 4 (23.53) |
| Preclerkship grades | 4 (23.53) |
| Shift to holistic review of students | 3 (17.65) |
| MD vs. DO degree discussion | 3 (17.65) |
USMLE, United States Medical Licensing Examination.
The comprehensiveness of each YouTube video was assessed based on the number of variables that were discussed. Each video was independently assessed by 3 authors for the presence of specific variable mentions. Microsoft Excel was used to mark the mention of the variables from each video. Cohen's kappa was used to determine the agreement between the authors. A score of 0.85 was calculated. Any discrepancies were resolved through joint video review and discussion until a consensus was reached.
The backgrounds of the YouTube content creators were analyzed to assess their qualifications to provide advice and on their credibility for speaking on orthopaedic surgery residency application advice, including factors such as speaker background (orthopaedics vs. nonorthopaedics), education level (attending, resident, or other) and speaker experience (residency PD, general advice, or personal anecdote, or general advice). Table II presents the demographic and mean view count data for the selected videos.
TABLE II.
Overview of Orthopaedic Residency Speaker Demographics on YouTube
| Demographics | No. of Videos (n = 17) | Mean No. of Views |
|---|---|---|
| Speaker background | ||
| Orthopaedic physician | 11 | 10,620 |
| Nonorthopaedic physician | 1 | 431,000 |
| Other | 5 | 2,850 |
| Race* | ||
| African American | 5 | 14,315 |
| Asian | 3 | 152,233 |
| Sex† | ||
| Male | 11 | 49,935 |
| Female | 2 | 3,410 |
| Mixed‡ | 4 | 1,191 |
| Experience | ||
| Admission committee | 4 | 34,563 |
| General residency advice | 2 | 18,350 |
| Personal anecdote | 11 | 8,229 |
| Education level | ||
| Attending | 10 | 54,693 |
| Resident | 3 | 739 |
| Student | 4 | 3,229 |
Statistics through self-identification from creator.
Determined using Gender Application Programming Interface Tool.
Implies both male and female speakers.
Gender representation among the content creators was determined using the Gender Application Programming Interface (API) (https://gender-api.com/), a leading gender inference service. Leveraging a large database of names, social media insights, and sociolinguistics, Gender API predicts sex based on a person's name—a methodology that has been tested, validated, and commonly used in representation studies across various fields8,9. However, it is important to note that this tool is restricted to binary gender classification and lacks the capability to accurately infer nonbinary genders. To supplement this, the authors documented each content creator's self-identified racial information to analyze minority and person of color (POC) representations. These data were extracted from YouTube channel descriptions, creators' previous videos, and alternative social media platforms. The distributions of race and sex among the content creators are summarized in Table II.
Results
A total of 17 orthopaedic surgery residency YouTube videos were evaluated, amassing 562,072 views at the time of the search. Table I indicates that the most mentioned variables were research experience and the USMLE Step 1 score (88.23%), personable skills (76.47%), and research years (64.71%). Variables such as MD or DO degrees and the shift to a holistic review of students were less prevalent, each discussed in 17.65% of videos. Letters of recommendation and the importance of interviews were discussed in 52.94% of videos. Table II shows that 11 videos, averaging 10,620 views each, were hosted by orthopaedic physicians, whereas a single video from a nonorthopaedic physician attracted 431,000 views. Five videos from creators self-identified as African Americans, with average views of 14,315 and 2,419, respectively. Three videos featured Asian-identifying creators, with an average view count of 152,233. Among the videos, 11 produced by male creators had an average of 49,935 views, while 2 by female creators had an average of 3,410 views. Videos featuring personal anecdotes (11 in total) averaged 8,229 views, whereas those from admission committee experiences (4 videos) averaged 34,563 views. Content creators who were attending physicians featured 10 videos with an average of 54,693 views, while resident-hosted videos (3 in total) averaged 739 views.
Discussion
Analysis of the selected YouTube videos revealed that research experience was the predominant variable discussed, mirroring the NRMP data reporting that accepted applicants typically have 3.8 research experiences: publications, poster presentations, or national conference participations2,3. The increased emphasis on research has prompted discussions around dedicating entire years to research projects which results in prolonging an applicant's career path10. This trend indicates that the expectation of academic research involvement is potentially being raised, influencing students to pursue more intensive research participation.
Many PDs foresee a heightened focus on research experiences in the wake of the USMLE Step 1 examination transitioning to a P/F grading system11. In the absence of a numerical Step 1 score to gauge applicant competitiveness, the extent of an applicant's engagement with research may emerge as a significant evaluative metric for residency programs. This anticipated shift could increase the number of applicants opting for research years, which could pose a financial challenge to those unable to afford a gap year. Furthermore, the advice given regarding the nature of research and the type of research preferred by PD's remains unclear. Applicants are presented with a broad spectrum of research pursuits. However, YouTube videos do not clarify whether orthopaedic residency programs prefer poster presentations, abstracts, publications, or specify the type of journals they favor. Neither do they provide guidance on how to acquire such research projects for medical students. The ambiguity of the advice may disadvantage students who lack direct access to orthopaedic mentors to assist in guiding them in gathering research experiences. There is a pressing need for explicit and pragmatic advice that addresses the type and quality of research that should be pursued.
Amid the recent transition of USMLE Step 1 to a P/F scoring system, available data regarding the implications of this shift remain limited. Nonetheless, it is predicted by various educational leaders and PDs that USMLE Step 2, which maintains its numerical scoring system, will gain increased attention in the application evaluation process11. Our analysis revealed that, while most videos highlighted the significance of Step 2 scores, the potential evolution of their relevance because of Step 1's transition was less emphasized. The 2022 NRMP PD Survey indicated that 47% of orthopaedic surgery residency programs require a target Step 2 score for interview consideration among US MD applicants3. This requirement is reported to be 35% for IMGs and 55% for DOs3. Because these dynamics may evolve over time, it becomes crucial for YouTube videos advising residency applicants to underscore the heightened importance of Step 2 scores. Notably, only 4 of the analyzed videos mentioned USMLE Step 1 scoring modification and its potential implications for the application process. This content gap represents another opportunity for future video creators to address an influential change.
Highlighting personal attributes and interpersonal skills as the third most frequently mentioned variable among the analyzed YouTube videos underscores their significance in the orthopaedic surgery residency application process. This focus aligns with the prevailing expectations in the field12. The 2022 NRMP PD Survey data further validate this, with 95% of PDs rating an applicant's personal attributes as a vital determinant in the selection process3. Moreover, the importance of an applicant's interpersonal skills, ethics, and professionalism emerged as the top-ranked quality among the discrete factors evaluated for applicants interested in orthopaedic surgery3. The recurring emphasis on such variables suggests that intrinsic qualities of an applicant are crucial. Therefore, the continuous emphasis on these variables in YouTube videos and in the current literature underscores the holistic approach to the selection of orthopaedic residents. Despite this trend, only 17.65% of the analyzed videos addressed this paradigm shift toward a holistic evaluation of applicants. This discrepancy reveals a significant opportunity for content creators to provide more extensive guidance on this topic.
Interestingly, only 52.94% of the videos analyzed discussed the importance of letters of recommendation and interview performance. This is particularly surprising, given their critical role in the residency selection process7. Letters of recommendation can provide unique insights into an applicant's work ethic and interpersonal skills.
The interview process is equally significant because it allows programs to assess similar attributes of a candidate that may not be entirely evident from written applications7. Bernatz et al. found that orthopaedic surgery PDs regarded the interview as the most influential factor when ranking and selecting applicants7. However, this crucial aspect of the application process is underrepresented in the analyzed YouTube videos. This discrepancy suggests that applicants relying on YouTube may not receive comprehensive guidance to effectively navigate interviews or secure impactful letters of recommendation. Content creators, therefore, have an opportunity to address these crucial topics more robustly.
Variables such as work experience, clerkship grades, class rank/Alpha Omega Alpha status, and personal statements were less frequently discussed in the analyzed videos, despite their significant importance in the orthopaedic surgery residency selection process3,5. These measures provide a snapshot of a candidate's potential for success in challenging fields. For example, personal statements are crucial because they allow candidates to showcase their motivation, commitment, and vision for their careers in orthopaedics. They offer a unique opportunity to communicate directly with the selection committee, providing a narrative that numbers alone cannot convey. Despite their potential influence on the selection process, these factors seem underrepresented in the YouTube videos analyzed. This information gap could leave potential applicants with a skewed or incomplete understanding of the application process.
Our analysis of speaker demographics in the evaluated YouTube videos offers insight into the credibility of online advice. Most videos (11/17) were created by orthopaedic physicians, potentially ensuring specialty-specific insights and advice for viewers. However, a single video by a nonorthopaedic physician garnered significantly more views (431,000) than those by orthopaedic physicians, suggesting that viewers may value general medical advice regardless of the speaker's specialty background. Further evaluation of the speaker backgrounds revealed that attending physicians dominated the videos (10/17). Nevertheless, considering the changing application process, medical students may also benefit from hearing from residents and students who have recently navigated the application process.
The current landscape of demographic representation in orthopaedic surgery is 75.9% White and 24.1% POC, ranking among one of the least diverse fields in ethnicity13. Furthermore, there was a predominance of male physicians within the specialty, comprising 93.5% of the field as of 201913. Among the videos analyzed, a more diverse racial distribution seems to be relatively diverse among the content creators. For instance, 5 videos featured self-identified African American creators, and 3 self-identified Asian creators. However, this distribution does not accurately represent the reported demographic data in orthopaedics, where 6.5% of orthopaedic physicians are African Americans, suggesting overrepresentation on YouTube.13
Given the diversity in the online representation of these videos, the potential under-representation of minorities remains a concern. For instance, only 2 videos were hosted by female creators. Such underrepresentation could limit the scope and inclusivity of the advice provided. In addition, 11 videos were presented by male creators, which often reflected their personal experiences. This is worth noting because there is potential for bias in the advice given, primarily reflecting the perspectives of male applicants. The lack of female and other gender-identifying speakers can limit the applicability and relevance of advice to applicants from diverse backgrounds.
Regarding the lens through which the advice was given, many videos (11/17) were based on personal anecdotes, which could help viewers understand the application process from different perspectives. However, the limited number of videos that offer insights from PDs indicates that viewers may benefit from diverse sources of information. While these first-hand accounts provide valuable insights into individual experiences, they may not capture the breadth of expert perspectives crucial for the application process. Thus, the integration of perspectives from both successful applicants and PDs can create a more balanced and comprehensive advice resource.
Building on these findings, we suggest that national orthopaedic organizations, such as the American Academy of Orthopedic Surgeons and the American Orthopedic Association, collaborate to create a comprehensive, sponsored video that tailors medical students interested in pursuing orthopaedic surgery. This video should include all necessary information from the latest application process changes to insights from recently successful applicants, residents, and current students, thus providing a multifaceted perspective on the pathway to orthopaedic surgery. Given the reputation and reach of these organizations, this initiative could effectively address the identified gaps and help ensure that students have access to accurate, current, and relevant guidance for navigating the orthopaedic surgery application process.
This study has several limitations. The search for relevant videos was conducted at a single point in time, which may not account for the new content produced during our data collection. Therefore, these findings may not capture all relevant videos available on the topic. YouTube's algorithm may also generate different video recommendations for users who use the same search terms, leading to potential variability in videos that arise from the search. YouTube's algorithm can affect the generalizability of our results because other users may have access to different resources based on their viewing histories and preferences.
Gender assessment relying on the Gender API is limited in its nonbinary identification capabilities, which could have led to an underrepresentation of these groups in this study. Moreover, it assumes a correspondence between names and sex that may not hold true in all cases, leading to potential inaccuracies in gender attribution. Similarly, the authors relied on self-identified racial information from content creators, which could have led to incomplete representation or misclassification because of the unavailability or inaccuracy of self-declared information.
Finally, the limited number of videos assessed may not be representative of the full range of advice available on YouTube. The accuracy of the information in these videos was not assessed, which is crucial for providing reliable advice. Given YouTube's variability in content comprehensiveness and credibility, the utility of advice for prospective orthopaedic surgery residency applicants may be affected. These limitations highlight the need for more nuanced and inclusive methods for identifying gender and racial diversity in future research.
Conclusion
This analysis of YouTube videos aimed at guiding orthopaedic surgery residency applicants revealed crucial gaps in content. While research experience is emphasized, other vital aspects, such as letters of recommendation, interview performance, and work experience, are underrepresented. This study underscores the need for comprehensive, sponsored content created in collaboration with national orthopaedic organizations to provide a thorough guide for prospective applicants.
Footnotes
Investigation performed at Michigan State University College of Human Medicine, Grand Rapids, MI
Disclosure: The Disclosure of Potential Conflicts of Interest forms are provided with the online version of the article (http://links.lww.com/JBJSOA/A569).
Contributor Information
Nicholas Jones, Email: jonesn37@msu.edu.
Mahfujul Z. Haque, Email: haquema4@msu.edu.
Zachary Jodoin, Email: zacharyjodoin@gmail.com.
References
- 1.Rynecki N, Para A, Gantz O, Coban D, Gala Z, Gupta R, Adeeb N, Ippolito JA, Ruskin J, Beebe KS. An analysis of trends in national residency matching program match data for orthopedic surgery. Orthopedics. 2020;43(6):e616-e622. [DOI] [PubMed] [Google Scholar]
- 2.Advanced Data Tables 2023. 2023. Available at: https://www.nrmp.org/wp-content/uploads/2023/04/Advance-Data-Tables-2023_FINAL-2.pdf. Accessed May 17, 2023. [Google Scholar]
- 3.NRMP (National Resident Matching Program). PD-Survey-Report-2022_FINALrev; 2022. Available at: https://www.nrmp.org/wp-content/uploads/2022/09/PD-Survey-Report-2022_FINALrev.pdf. Accessed July 29, 2023.
- 4.Rehman R, Ahmed L, Kohen L. YouTube as a source of dermatology residency information. Int J Dermatol. 2023;62(4):e233-e234. [DOI] [PubMed] [Google Scholar]
- 5.O'Mary K, Tompane T, Perry N, Smith J, Janney C. Matching into orthopedic surgery in the U.S. Navy: a review of applicant selection criteria. Mil Med. 2022:usac267. [published online ahead of print, 2022 Sep 9]. doi: 10.1093/milmed/usac267. [DOI] [PubMed] [Google Scholar]
- 6.Mun F, Scott AR, Cui D, Lehman EB, Jeong S, Chisty A, Juliano PJ, Hennrikus WL, Hennrikus EF. A comparison of orthopaedic surgery and internal medicine perceptions of USMLE Step 1 pass/fail scoring. BMC Med Educ. 2021;21(1):255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Bernatz JT, Johnson KP, Stokman JJ, Cannada LK, Williams SK, Whiting PS. Factors considered in ranking spine surgery fellowship applicants: a Survey of program directors. Spine. 2021;46(13):882-5. [DOI] [PubMed] [Google Scholar]
- 8.Lee SF, Redondo Sánchez D, Sánchez MJ, Gelaye B, Chiang CL, Wong IOL, Cheung DST, Luque Fernandez MA. Trends in gender of authors of original research in oncology among major medical journals: a retrospective bibliometric study. BMJ Open. 2021;11(10):e046618. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hupalo S, Martinowich K, Carlezon WA, Jordan CJ. Neuropsychopharmacology (NPP) 2020 report on gender balance among corresponding authors and reviewers: before and during the COVID-19 pandemic. Neuropsychopharmacology. 2022;47(5):973-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wright-Chisem J, Cohn MR, Yang J, Osei D, Kogan M. Do medical students who participate in a research gap year produce more research during residency? J Am Acad Orthop Surg Glob Res Rev. 2021;5(5):e21.00061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Goshtasbi K, Abouzari M, Tjoa T, Malekzadeh S, Bhandarkar ND. The effects of pass/fail USMLE Step 1 scoring on the otolaryngology residency application process. Laryngoscope. 2021;131(3):E738-E743. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Finkler ES, Fogel HA, Kroin E, Kliethermes S, Wu K, Nystrom LM, Schiff AP. Factors influencing the number of applications submitted per applicant to orthopedic residency programs. Med Educ Online. 2016;21(1):31865. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Peterman NJ, Macinnis B, Stauffer K, Mann R, Yeo EG, Carpenter K. Gender representation in orthopaedic surgery: a geospatial analysis from 2015 to 2022. Cureus. 2022;14(7):e27305. [DOI] [PMC free article] [PubMed] [Google Scholar]