Glaucoma and Machine Learning: A Call for Increased Diversity in Data

The COVID-19 pandemic has shed light on long-standing health inequities and systemic racism, with Black and Hispanic communities suffering disproportionately across the US. These disparities are seen across different conditions and glaucoma is no exception. In the United States, glaucoma disproportionately affects racial minorities. We know that race/ethnicity modulates risk for glaucoma significantly; Black individuals are four to five times more likely to have glaucoma than White individuals.¹ Black patients with glaucoma are also more likely to develop vision impairment or blindness.² Similarly, Hispanic individuals are two to three times more likely to have glaucoma than White individuals and are estimated to have a higher undiagnosed case rate compared to their White counterparts.^3–5 As medicine becomes progressively predictive and personalized, these disparities in risk and severity of disease must be considered.

These statistics likely represent a complex interplay of socioeconomic, political (e.g., systemic racism) and biologic variables. This, together with the complex and clinically heterogenous nature of the disease, make glaucoma an ideal candidate for individualized and precision medicine approaches to better screen, diagnose, and monitor progression. The diagnosis and determination of glaucoma disease progression relies on data, in particular imaging methods such as fundus photographs and optical coherence tomography (OCT) scans. In recent years, machine learning, a highly effective tool for automated image processing, has been increasingly researched and applied extensively in glaucoma (Figure 1). Despite the known racial/ethnic disparities in glaucoma outcomes, imaging databases used to study and characterize glaucoma in machine learning approaches do not have appropriate representation of Black and Hispanic individuals who are most at risk.

Figure 1.

Graph representing the number of articles researching machine learning as a tool for automated image processing in glaucoma.

Practicing equitable care in the age of precision medicine requires us to have a keen understanding of how each individual is affected by disease. While machine learning can be used to screen and diagnose glaucoma, we must include and accurately represent characteristics that modulate disease risk, such as race/ethnicity, in the data used to train models. However, this is not currently the standard practice. For example, large population-based studies with imaging data such as the UK Biobank are currently being used heavily in exploratory research. These studies primarily include data from a majority European or of European descent participants.^6–9 Similarly, a review of publicly available datasets for ophthalmological imaging found that 75 of the 94 open access datasets are from patients in Asia, North America, and Europe, with little representation of patients from low- and middle-income countries.¹⁰ 74% of these datasets did not report any demographic data, like age, sex, and race/ethnicity.¹⁰

To better understand how demographically representative the latest machine learning glaucoma research has been, we reviewed recent glaucoma and machine learning publications, using PubMed search terms including “glaucoma” and “machine learning” and also included publications selected based on expert knowledge. We included manuscripts that used machine learning on images (fundus photographs, OCT scans) to classify glaucoma or characterize glaucomatous structures and excluded studies with fewer than 500 participants or published before 2016. Of the latest literature reporting on glaucoma and machine learning, 59% did not report race and/or ethnicity (Figure 2). Of the 11 studies with patient data solely from North America, we found that only 6 reported race and/or ethnicity. Of these 6 studies, 4 used data from the Duke Glaucoma Registry and had between 17.7% - 43.2% “African-American” representation.^11–14 The other 2 studies report 22% - 45.5% “Black” or “Black/African descent” representation in various samples of their data.^15,16 Only one study reported inclusion of a Hispanic population; this study had only 1.5% Hispanic representation.¹¹ The remaining 10 studies had patient data from Asia, Europe, or multiple continents; only 3 of those studies reported race and/or ethnicity (Figure 2).

Figure 2.

Graph representing the literature on glaucoma and machine learning in articles reporting race and/or ethnicity.

While the field of glaucoma and machine learning is progressing, our brief review demonstrates that the imaging databases currently used to fuel its research are not representative of the population in the US and importantly the populations most at risk of disease. Most studies from the US do not even report race/ethnicity data; the few that do report such data and include proportionate representation of Black populations were largely from a single dataset. While our findings may not be surprising given the historically biased nature of large datasets in ophthalmology, they certainly are problematic.

Developing and training experimental machine learning models on a limited or unknown population may decrease the generalizability of these algorithms for glaucoma detection. Glaucoma has a faster onset, more severe presentation, and more rapid progression in Black populations and remains relatively understudied in Hispanic populations.¹⁷ There are also differences in optic disc size and shape, cup-to-disc ratios, vascular patterns, and retinal appearance amongst populations of different races and ethnicities, as well as disparities in socioeconomic status, access to healthcare, and exposures to systemic racism which may accelerate biological aging.^18–23 Given these differences, we believe that training models on images in White populations may lead to more missed diagnoses and poorer care in Black and Hispanic populations.

Indeed, biased data has already been shown to limit predictive power of machine learning algorithms in broader populations. Prediction models for assessing risk of coronary heart disease based on the Framingham Heart Study, a study of White middle-class individuals, performed poorly in Japanese, Hispanic, and Native-American populations.²⁴ Another study showed that commercial prediction algorithms trained in majority White populations incorrectly underestimated Black patients’ healthcare “risk” scores, in comparison to equally sick White patients.²⁵ Similarly, a deep learning model trained on fundus photographs from primarily European-descent and Asian-descent populations performed poorly for detection of glaucoma in a primarily African-descent population.¹⁸ Conversely, when algorithms are trained on balanced and representative datasets they perform well. For example, an algorithm to diagnose common skin conditions that was trained on a dataset with appropriate representation of minority groups had similar performance across different races.²⁶ Another study found that a deep learning model trained on knee X-rays with appropriate representation of Black patients was able to reduce previously unexplained racial disparities in pain scores seen in osteoarthritis.²⁷

Current databases widely used in machine learning research for glaucoma are under representative of minority groups - those who are at the highest risk for severe glaucoma. It is clear that this approach can negatively impact the predictive power of precision medicine approaches for screening and diagnosis of glaucoma in these at-risk populations. This in turn will compound already poorer outcomes for Black and Hispanic communities across the US. As machine learning in glaucoma progresses, we call for an urgent commitment to increase the diversity of datasets and improve race/ethnicity reporting. This is a crucial step towards halting the propagation of inequities and ensuring just and quality care for all patients suffering from glaucoma.