This quality improvement study assesses for the addition of diversity or inclusion information on US ophthalmology residency program websites.
Key Points
Question
What proportion of US ophthalmology residency program websites have diversity or inclusion information on their website?
Findings
In this quality improvement study, a total of 29 of 121 US ophthalmology residency program websites (24%) met at least 1 of 6 predetermined diversity or inclusion information criteria.
Meaning
Although this investigation cannot determine how an ophthalmology residency program's website reflects the diversity and inclusion within the program, the results suggest that most program websites lack information regarding their support, value of, or efforts toward increasing diversity or inclusion in their program.
Abstract
Importance
Groups historically underrepresented in medicine also remain underrepresented within ophthalmology. Prospective applicants, in particular women and racial and ethnic minority groups, consider program factors related to gender diversity, ethnic diversity, and program culture more than nonminority groups. A prospective applicant’s decision to apply to a particular residency program may be influenced by the information presented on the program’s website.
Objective
To evaluate for presence of diversity or inclusion information on US ophthalmology program websites.
Design, Setting, and Participants
This quality improvement study, conducted from May 1 to June 2, 2021, included a cross-sectional review of ophthalmology program websites. Ophthalmology program websites were reviewed for the presence of 6 predetermined diversity or inclusion information criteria.
Main Outcomes and Measures
The presence of 6 predetermined diversity or inclusion information criteria. Association between program size and ranking with presence of diversity or inclusion information criteria was assessed.
Results
Of the 121 US program websites reviewed, 29 (24%) met at least 1 of the 6 predetermined criteria. Programs with more than 12 residents (odds ratio [OR], 3.58; 95% CI, 1.39-9.55; P = .004) and programs ranked within the top 20 best hospitals for ophthalmology (OR, 3.28; 95% CI, 1.05-10.1; P = .02) were more likely to have at least 1 of the diversity or inclusion information criteria on their website.
Conclusion and Relevance
Results of this quality improvement study suggest that the majority of US ophthalmology residency programs do not have information regarding diversity or inclusion on their websites. Although this investigation cannot determine how an ophthalmology residency program's website reflects the diversity and inclusion within the program, these findings suggest that there is a gap in the information that prospective applicants can collect when evaluating ophthalmology residency program websites.
Introduction
Despite the expected growth of minority populations in the US,1 women and groups traditionally underrepresented in medicine—self-reported Black, Hispanic/Latino, Native American, Native Hawaiian, and Pacific Islander individuals—remain underrepresented among practicing ophthalmologists, ophthalmology faculty, and residents when compared with the general population.2 In a conscious effort to increase diversity in the field, the Accreditation Council for Graduate Medical Education updated the Common Program Requirements to advocate for residency programs’ practices that actively focus on recruiting and retaining a diverse and inclusive workforce.3
Portraying the commitment that residency programs aim to provide to diversity and inclusion efforts may be particularly important to attract a diverse pool of applicants, especially students historically underrepresented in the field, such as women, self-reported racial and ethnic minority groups, and lesbian, gay, bisexual, transgender, and queer applicants. When ranking residency programs, women and self-reported racial and ethnic minority groups, in particular, are known to consider program factors related to gender and ethnic diversity and inclusion and program culture more than nonminority groups.4,5 Caring for an underserved population and diversity and inclusion of staff and faculty have also been factors that applicants typically consider when applying to ophthalmology residency programs.6 Furthermore, a prospective applicant’s decision to apply to a particular program may be influenced by the information presented on the programs’ website.7 As such, program websites deemed to be incomplete by applicants present a barrier to their information needs.8 We believe that perhaps diversity and inclusion information is one of the required factors for completeness.
This is perhaps even more relevant given the substantial effect that the COVID-19 pandemic had on the 2020-2021 match experience. Residency interviews were changed to a virtual format to promote safety and ensure that applicants and residency programs from affected regions were not disadvantaged by their inability to attend or host in-person interviews.9 In light of the presenting challenges, the Association of University Professors of Ophthalmology created a database to collect and provide prospective applicants with accurate ophthalmology residency program data in a standard fashion.9 None of the provided data points included information regarding diversity or inclusion.
A previous study evaluating the presence of diversity or inclusion promotion among US Physical Medicine and Rehabilitation program websites found that only a small number of residency programs included any diversity or inclusion information on their websites.10 To our knowledge, no previous study has described diversity or inclusion information presence in ophthalmology residency program websites. The purpose of this study was to evaluate for the presence of predetermined diversity or inclusion information and materials in US ophthalmology residency program websites.
Methods
This quality improvement study was conducted from May 1 to June 2, 2021. The Weill Cornell Medicine institutional review board deemed this research to be exempt from review as this study did not constitute human subject research. The Standards for Quality Improvement Reporting Excellence (SQUIRE) reporting guidelines were followed for this study.
Study Goal and Aims
The primary goal of this study was to evaluate for and describe the presence of diversity or inclusion materials in US ophthalmology residency program websites. Our primary aim was to determine the proportion of program websites with at least 1 of 6 predetermined diversity or inclusion information criteria. We expected that a majority of the programs’ websites would not have any of the criteria present on their website.
The criteria, modeled after those implemented in previously published literature,11 included (1) mention of commitment to, or value toward, diversity in the residency program mission statement, program director’s message, or department chair’s message, (2) a separate diversity mission statement (stand-alone statement of the same commitment elsewhere on the site), (3) mention of rotations or fellowship opportunities for underrepresented minority medical students, (4) mention of diversity initiatives (any resource dedicated to promoting diversity or inclusion within a program aside from diversity in mission statement, ie, diversity council, diversity groups, calendar, newsletter, certificate program, etc), (5) a diversity page or section, and (6) appointed diversity leadership position(s). These criteria have not been previously validated for this purpose.
We further evaluated whether program size (more or less than 12 residents) or program ranking (top 20 vs non–top 20) was associated with presence of the criteria.
Data Collection
A complete list of the 125 US ophthalmology residency programs was obtained from the Fellowship and Residency Electronic Interactive Database (FREIDA) in May 2021. A total of 121 program website links were obtained through FREIDA or Google Search and were included in the analysis. Four residency programs were excluded owing to lack of a program website. Any information not readily available or linked to the main residency or department website page was not considered in the analysis. Videos on program websites were not part of the analysis.
Program size (number of postgraduate year 2 to postgraduate year 4 residents) was collected from each program website. Program ranking was obtained from the list of “Best Hospitals for Ophthalmology” per the 2020 US News & World Report.12
Statistical Analysis
Cross-sectional and descriptive analysis of the collected data were performed. Fisher exact tests were used to compare proportions. Reported P values were 2-sided with statistical significance evaluated at the .05 α level. P values were not adjusted for multiple analyses. To assess the precision of the obtained estimates, 95% CIs for parameters of interest were calculated. Statistical analyses were conducted in R, version 1.4.1717 (R Studio).
Results
Of the 121 program websites reviewed, 29 (24%) met at least 1 of the 6 predetermined diversity criteria, and 92 (76%) did not meet any of the criteria. Table 1 illustrates the number of diversity or inclusion criteria met on the program websites. A total of 42 programs (35%) had more than 12 residents.
Table 1. Diversity or Inclusion Criteria Met on US Ophthalmology Residency Program Websites.
| Diversity or inclusion criteria | No. (%) |
|---|---|
| Total No. | 121 |
| Any of the criteria: | 29 (24) |
| Mention of diversity in mission statement, program director’s statement, or department chair’s message | 9 (7) |
| Stand-alone diversity statement | 17 (14) |
| Minority rotation/fellowship | 9 (7) |
| Diversity | |
| Initiatives | 11 (9) |
| Page/section | 7 (6) |
| Leadership position | 6 (5) |
In a univariate analysis, we found that programs were more likely to meet at least one of the diversity or inclusion information criteria if they had more than 12 residents (odds ratio [OR], 3.58; 95% CI, 1.39-9.55; P = .004) or were ranked within the top 20 “Best Hospitals for Ophthalmology” (OR, 3.28; 95% CI, 1.05-10.1; P = .02). Four programs met 5 or more of the predetermined diversity or inclusion information criteria: the University of California in San Diego, Stanford University, Stanford, California, Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, and University of Utah, Salt Lake City. Two programs met 4 criteria, 7 programs met 2 criteria, and 16 programs met only 1 of the criteria (Table 2).
Table 2. Number of Criteria Met by Ophthalmology Residency Program Websites Meeting Any Criteria (N = 29).
| No. of criteria met | No. of programs, No. (%) |
|---|---|
| 1 | 16 (55) |
| 2 | 7 (24) |
| 3 | 0 (0) |
| 4 | 2 (7) |
| 5 | 3 (10) |
| 6 | 1 (3) |
Aside from the predetermined criteria, we observed that 3 program websites displayed a clear statement against racism. One of these programs had not met any of the predetermined criteria. One program website had a section about race and bias in health care and explicitly described the importance of holistic reviews in their program. Another program website displayed a specific set of actions to take in order to foster diversity in the field and provided a racial equity glossary. Additionally, we observed that 53 of the program websites (44%) mentioned that their residency program offers the opportunity to care for a diverse population.
Discussion
There has been a noted effort of residency programs across all specialties to optimize their online presence.11 Though the content of ophthalmology residency program websites has previously been evaluated for comprehensiveness,13 no previous study, to our knowledge, had evaluated for presence of diversity or inclusion criteria. In this quality improvement study, we found that only 24% of evaluated program websites had any mention of commitment to, or value toward, diversity in the form of statements, opportunities for underrepresented minority groups, program initiatives, or appointed diversity leadership. The diversity and inclusion information presented on a program website need not be limited to the 6 criteria used here. For example, a statement against racism, as observed in some program websites, could suggest a culture of inclusion. An important next step of this study will be to evaluate what information specifically regarding diversity and inclusion is valuable to prospective applicants and the effectiveness of this information in diversity recruitment efforts.
Limitations
This study had some limitations. The presence of the evaluated criteria on program websites does not necessarily reflect the diversity or inclusion of the residency program. The predetermined criteria have not been validated for the purposes of this study. No adjustment to P values was made for multiple analyses. Additionally, programs may not have updated their sites at the time of this study, and/or residency program leadership may not have control over website content. In addition, the presence of the evaluated criteria, such as information about rotations or fellowships for minority applicants, is likely to be influenced by program funding and resources, which varies across institutions.
The significant positive association of top 20 rank and program size with likelihood of meeting at least one of the predetermined criteria warrants further investigation. The use of the US News & World Report for ranking, although subjective, was the only publicly available national ranking system.
Importantly, restructuring a programs’ website is just one of the potential strategies programs can use to facilitate recruitment of racial and ethnic minority groups.14 This change on its own will not necessarily lead to increased diversity or inclusion within a program but may make a program’s commitment to diversity and inclusion more apparent.
The objective of this study aligns with recent calls for an improved and more transparent application process for aspiring ophthalmologists of all backgrounds.15 We recognize the ever-changing landscape of graduate medical education, and we present this observed gap in diversity and inclusion information as an opportunity for improvement in what programs present to prospective ophthalmology applicants in an era where the internet is a main resource for information.
Conclusions
Results of this quality improvement study suggest that most US ophthalmology residency programs do not have information regarding diversity or inclusion on their websites. This represents a gap in the information that prospective applicants can obtain when evaluating ophthalmology residency program websites. Ultimately, actionable change toward increasing and fostering diversity, inclusion, and equity in programs’ training environments is likely to be the most effective strategy.
References
- 1.Colby SL, Ortman JM. Projections of the size and composition of the US population:2014-2060. Accessed May 6, 2021. https://www.census.gov/content/dam/Census/library/publications/2015/demo/p25-1143.pdf
- 2.Xierali IM, Nivet MA, Wilson MR. Current and future status of diversity in ophthalmologist workforce. JAMA Ophthalmol. 2016;134(9):1016-1023. doi: 10.1001/jamaophthalmol.2016.2257 [DOI] [PubMed] [Google Scholar]
- 3.Accreditation Council for Graduate Medical Education . The program director guide to the common program requirements (residency). Accessed May 6, 2021. https://dl.acgme.org/learn/course/the-program-directors-guide-to-the-common-program-requirements-residency-ebook/interactive-handbook/e-book
- 4.Agawu A, Fahl C, Alexis D, et al. The Influence of gender and underrepresented minority status on medical student ranking of residency programs. J Natl Med Assoc. 2019;111(6):665-673. doi: 10.1016/j.jnma.2019.09.002 [DOI] [PubMed] [Google Scholar]
- 5.Ku MC, Li YE, Prober C, Valantine H, Girod SC. Decisions, decisions: how program diversity influences residency program choice. J Am Coll Surg. 2011;213(2):294-305. doi: 10.1016/j.jamcollsurg.2011.04.026 [DOI] [PubMed] [Google Scholar]
- 6.Yousuf SJ, Kwagyan J, Jones LS. Applicants’ choice of an ophthalmology residency program. Ophthalmology. 2013;120(2):423-427. doi: 10.1016/j.ophtha.2012.07.084 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ. Aspects of residency programs’ web sites important to student applicants. Acad Emerg Med. 2005;12(1):89-92. doi: 10.1197/j.aem.2004.08.047 [DOI] [PubMed] [Google Scholar]
- 8.Embi PJ, Desai S, Cooney TG. Use and utility of web-based residency program information: a survey of residency applicants. J Med Internet Res. 2003;5(3):e22. doi: 10.2196/jmir.5.3.e22 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Quillen DA, Siatkowski RM, Feldon S; of the Association of University Professors of Ophthalmology . COVID-19 and the ophthalmology match. Ophthalmology. 2021;128(2):181-184. doi: 10.1016/j.ophtha.2020.07.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Sanchez AN, Martinez CI, Lara AM, Washington M, Escalon MX, Verduzco-Gutierrez M. Evaluation of diversity and inclusion presence among US physical medicine and rehabilitation residency program websites. Am J Phys Med Rehabil. 2021;100(12):1196-1201. doi: 10.1097/PHM.0000000000001693 [DOI] [PubMed] [Google Scholar]
- 11.Tang OY, Ruddell JH, Hilliard RW, Schiffman FJ, Daniels AH. Improving the online presence of residency programs to ameliorate COVID-19's impact on residency applications. Postgrad Med. 2021;133(4):404-408. doi: 10.1080/00325481.2021.1874195 [DOI] [PubMed] [Google Scholar]
- 12.US News & World Report . Best hospitals for ophthalmology. Accessed May 10, 2021. https://health.usnews.com/best-hospitals/rankings/ophthalmology
- 13.Shaath DS, Whittaker TJ. Evaluation of Ophthalmology Residency Program Web Sites. J Acad Ophthalmol. 2019;11(2):e44-e48. doi: 10.1055/s-0039-3400736 [DOI] [Google Scholar]
- 14.Harrell M Jr, Barnett KG, Rowe S. Strategies potentially associated with increasing racial and ethnic groups underrepresented in medicine: application to ophthalmology. JAMA Ophthalmol. 2021;139(8):825-826. doi: 10.1001/jamaophthalmol.2021.2124 [DOI] [PubMed] [Google Scholar]
- 15.Liebman DL, Armstrong GW, Shah AS, Lorch AC, Miller JW, Chodosh J. The case for transparency in the ophthalmology residency match. Ophthalmology. 2021;128(2):185-187. doi: 10.1016/j.ophtha.2020.10.019 [DOI] [PubMed] [Google Scholar]
