Addressing racial disparities in surgical care with machine learning

Abstract

There is ample evidence to demonstrate that discrimination against several population subgroups interferes with their ability to receive optimal surgical care. This bias can take many forms, including limited access to medical services, poor quality of care, and inadequate insurance coverage. While such inequalities will require numerous cultural, ethical, and sociological solutions, artificial intelligence-based algorithms may help address the problem by detecting bias in the data sets currently being used to make medical decisions. However, such AI-based solutions are only in early development. The purpose of this commentary is to serve as a call to action to encourage investigators and funding agencies to invest in the development of these digital tools.

Racial disparities in surgical care are a well-documented reality across the United States. Black patients receiving cardiac surgery have 17% and 26% higher odds of mortality and major postoperative complications, respectively, compared to White patients¹. In spine surgery, the risk of postoperative complications has been estimated to be as much as 61% higher for Black patients. And it should be noted that these estimates were risk-adjusted for comorbidities, hospital characteristics, baseline patient status, and other factors, with the unadjusted discrepancies being even wider. The fact that Black patients receive a lower quality of care has been a known fact for over four decades^2–4, but despite several efforts to address the problem^5–7, the gap is far from closed⁸.

Access to health care and barriers

One of the main reasons for the inequality in surgical outcomes is unequal access to health care. This lack of access has three main components: decreased exposure to preventive practices, lower rates of health care utilization, and delayed presentation. Black patients, for instance, are less likely to receive routine cancer screening and tend to present later for the management of preventable or early-detectable cancers, such as cervical or colorectal^9,10. They are also less likely to receive hip and knee arthroplasty, lumbar surgery, carotid endarterectomy, and others¹¹.

Although timely surgical intervention often results in better clinical outcomes, pathologies that are allowed to progress are less likely to respond to management. Several of the aforementioned factors—insurance, health literacy, economic status— may eventually result in Black patients presenting later in the natural timeline of their disease, restricting the benefit they may obtain from surgery. Among lung cancer patients of I–IIIA stage, for instance, Neroda et al. found that Black individuals were almost 70% more likely to receive delayed surgery—defined as a time from diagnosis to surgery of more than 6 weeks in this study. Similar findings have been reported in spine, benign brain tumor, and hip replacement surgery, with the delayed presentation being a mediator towards worse postoperative outcomes in Black patients^12–14. Overall, hindered access to health care makes Black patients worse surgical candidates upon presentation.

It is no surprise to find that poor insurance coverage often contributes to the under-utilization of health care among Black individuals. Most studies supporting this observation have investigated the differences between private insurance, Medicare and Medicaid coverage, and lack of any insurance, and have found them to be significant. Black individuals are more prone to lose health care coverage at any point in their lives, recording a proportion of uninsured person-years of 0.20, compared to 0.12 among White individuals. Private insurance is associated with easier appointment scheduling¹⁵, delivery of more patient-friendly care practices— such as minimally invasive and outpatient surgery^16,17—and superior surgical outcomes when compared to government payors and a lack of insurance coverage^18,19. However, the comparison of private-payer programs, government-issued programs, and lack of insurance illuminates only part of the story, as there is a large heterogeneity among private insurance programs that could potentially impact access to health care. More specifically, higher-deductible plans discourage patients from pursuing contact with a provider and are more prevalent among Black individuals²⁰.

Health literacy, prior individual experiences, and cultural traits may also contribute to the health disparities between Black and White individuals. Ibrahim et al. conducted a survey on patients with hip or knee osteoarthritis to assess the patients’ heuristics and expectations from care. They found that Black patients were more inclined towards complementary or self-administered therapeutic options, while they were less likely to consider joint replacement surgery²¹. Research also suggests that less education contributes to this phenomenon²². In addition, Black patients are more likely to overestimate the length of hospitalization, procedure-related pain, and disability; overall they are more skeptical about joint replacement surgery than White patients²³.

Comparing delivered quality of care

Another potential component of the racial disparities in surgical care is the discrepancies in the quality of care delivered. In a study by Rangrass et al. utilizing a national claims database, it was shown that hospital quality might explain as much as 35% of the observed discrepancy in mortality after coronary artery bypass graft surgery between Black and White patients²⁴. However, this conclusion is refuted by Silber et al., who utilized the same database to study the same hypothesis on general surgical procedures. The investigators of the latter study found that discrepancies in outcomes were eliminated following matching Black and White patients on preoperative status; hence, they suggested that racial disparities should be attributed to the delayed access to care rather than the heterogeneity of care quality among providers and institutions²⁵.

Addressing surgical disparities

Several federal programs and scientific community initiatives have been launched to address racial disparities in health care during the past two decades^5–7. In 2011, the Department of Health and Human Services announced a multidimensional plan to address the racial gap in healthcare: this plan included policy modifications, funding redistribution, and rewards for the care of socially disadvantaged populations, among others. Subsequently, the Affordable Care Act provided poorer individuals with enhanced insurance options. Buchmueller et al. found the Affordable Care Act to lower the uninsured rate among Black individuals by almost 35%. Nevertheless, overall, these measures have not been as impactful as desired, and the landscape remains essentially unchanged. In a study using the National Inpatient Sample, Best et al.⁸ investigated the utilization of nine common procedures by race relative to the proportion of races in the population and found the gap between them having become smaller in some cases, while larger in others, while at no point reaching equality between Black and White individuals. It is clear that racial disparities are still an unresolved problem in society, and the high-level policies employed so far have not proven sufficient to eradicate them.

Can surgical bias yield to AI-based algorithms?

It is unrealistic to imagine that discrimination against various population subgroups can be resolved with artificial intelligence alone. The cultural, ethical, and sociological issues are far too complex to solve with digital tools, regardless of how sophisticated they may be. Nonetheless, AI and machine learning that addresses a variety of technological touch points can improve the profession’s ability to detect bias and improve patients’ access to surgical services and outcomes.

To ensure that all patients are ensured equal access to high-quality medical care, including surgical services, it is first necessary to analyze the data sets used to determine whether patients of color, women and those in lower socioeconomic groups are accurately represented in the data sets and algorithms used to determine the need for said services. As we have pointed out in a previous publication²⁶, this has not always been the case. Obermeyer et al.’s²⁷ analysis of a commercial database has demonstrated that, while Blacks were considerably sicker than White patients, based on signs and symptoms, the dataset did not recognize the greater disease burden in Blacks because it assigned risk scores based on total healthcare costs accrued. It is unrealistic to assume that such costs accurately measured patients’ needs; the lower cost among Blacks may have been due to less access to care, which in turn resulted from their distrust of the healthcare system and direct racial discrimination from providers²⁸. Similar discrimination against women has been documented in medical imaging datasets used to train and test AI systems used for computer-assisted diagnosis²⁹. There is also evidence to suggest that some machine learning enhanced algorithms that rely on electronic health record data under-represent patients in lower socioeconomic groups³⁰.

Commercially available AI bias detection tools that have been used to help identify discrimination include concept activation vectors (TCAV), which are used by Google to measure bias by race, gender, and location³¹, and Audit-AI, which uses a Python library from Pymetrics that can detect discrimination by locating specific patterns in the training data^26,32.

Devising a comprehensive bias detection toolkit

While the aforementioned bias detection programs have merit, solving the problem of surgical bias will require a more comprehensive approach. That approach begins with a set of guidelines that set forth standards on how to conduct AI-related research and how to report it in the professional literature, including The Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence (SPIRIT-AI)extension, a set of guidelines designed to help researchers develop AI-related clinical trials³³, and the Consolidated Standards of Reporting Trials-Artificial Intelligence (CONSORT-AI) extension³⁴ Unfortunately, despite the recommendations from thought leaders regarding the importance of adhering to standards that would make algorithms more equitable, Lu et al. have found these guidelines are often ignored³⁵.

They looked at 15 model reporting guidelines and reviewed 12 deployed Epic models. They found a median completion rate was 39% and stated: “…information on usefulness, reliability, transparency, and fairness was missing from at least half of documentation.”

Mayo Clinic is taking a more direct approach to algorithmic bias. Mayo Clinic Platform (MCP) has developed _Validate, a digital solution that helps measure model sensitivity, specificity, area under the curve (AUC), and bias, which in turn enables the system to break down the racial, gender, and socio-economic disparities in the delivery of care. Using the tool can lend credibility to models, accelerates adoption into clinical practice, and enables developers to more readily meet regulatory requirements for approval. It provides users with a series of descriptive statistics of model performance and data to demonstrate that the model was run against each demographic.

To illustrate _Validate’s performance, imagine that a developer wants to create a clinical solution that predicts whether a patient with signs and symptoms of appendicitis will need surgery or can be managed with antibiotics. Inputs fed into the algorithm might include all historical patient data, including demographics, prior diagnoses, a history of abdominal abnormalities, and family history of the same. _Validate would provide testability that has been missing from many commercially available products. It enables health care stakeholders to test an AI model against an extensive data set and evaluate the reasonableness and usefulness of the result. In addition to its ability to evaluate and certify the quality and accuracy of an AI model, _Validate protects the intellectual property of the model and its data, using state-of-the-art de-identification protocols. With the assistance of Diagnostic Robotics, a validation services provider, _Validate analyzes the model’s performance, generating a table that includes true negatives, false negatives, true positives, and false positives, from which sensitivity, specificity, AUC, and positive predictive and negative predictive values can be derived. It can also perform a biased evaluation that takes into account race, ethnicity, age, obesity, behavioral health, genetic history, gender, and socioeconomic status markers.

Johns Hopkins University investigators are also taking measures to solve the bias problem. Wang et al have developed an 11-question checklist to help assess the validation of predictive models³⁶. Among the issues that the checklist asks algorithm developers to take into consideration:

• “Is the prediction target an appropriate proxy for patient health care outcomes or needs?”

• “Are there any modeling choices made that could lead to bias? For example, are there any dependencies between inputs and outcomes that could lead to discriminatory performance across groups?”

• “Was the data used to train the model representative of the population in the deployment environment?”

• “Do validation studies report and address performance differences between groups?”

Innovation depends upon a perfect storm of technology, policy, and culture. Machine learning techniques, including deep learning systems, are mathematically robust in 2022 and commercially supported by all cloud providers, so it is fair to say that technology is not a rate-limiting step. Policies for the guardrails and guidelines of the machine learning life cycle to reduce bias and monitor ongoing fairness and usefulness, on the other hand, are still a work in progress; Several of us have assembled a multi-stakeholder coalition (coalitionforhealthai.org) to provide the foundational implementation guides that may evolve into policy. Culture likewise will require additional focus. We must set a cultural expectation that machine learning in healthcare should only be deployed in production when equity is a design principle. Finally, we believe that machine learning is only one tool in our quiver to reduce racial disparities in surgery, but it can be rapidly deployed, locally optimized, and monitored for impact over time.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Author contributions

J.H., M.B., P.C., and A.B. collected the data; conceived, designed, and performed the analysis; reviewed the literature; and wrote the paper.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41746-022-00695-6.