Leveraging large language models to foster equity in healthcare

Abstract

Objectives

Large language models (LLMs) are poised to change care delivery, but their impact on health equity is unclear. While marginalized populations have been historically excluded from early technology developments, LLMs present an opportunity to change our approach to developing, evaluating, and implementing new technologies. In this perspective, we describe the role of LLMs in supporting health equity.

Materials and Methods

We apply the National Institute on Minority Health and Health Disparities (NIMHD) research framework to explore the use of LLMs for health equity.

Results

We present opportunities for how LLMs can improve health equity across individual, family and organizational, community, and population health. We describe emerging concerns including biased data, limited technology diffusion, and privacy. Finally, we highlight recommendations focused on prompt engineering, retrieval augmentation, digital inclusion, transparency, and bias mitigation.

Conclusion

The potential of LLMs to support health equity depends on making health equity a focus from the start.

Introduction

Emerging technologies are transforming healthcare, but will they help close health disparities or exacerbate them? Two key currents in healthcare are a growing focus on health equity and the rise of large language models (LLMs). These currents interact and may be either synergistic or in opposition. Healthcare organizations are acknowledging the drivers of disparities, like social and structural determinants of health. At the same time, LLMs, which are artificial intelligence (AI) systems capable of processing and generating human-like text, are being lauded and integrated into care.¹ However, their impact on health equity is not yet clear. Historically, marginalized populations have been left behind by new technologies and experience bias propagated by AI.² If LLMs are to have a net positive impact on health equity, we must prioritize the use of LLMs for health equity efforts and consider how LLMs can be deployed equitably. In this perspective, we use the National Institute on Minority Health and Health Disparities (NIMHD) research framework, which highlights the factors driving health disparities, to discuss opportunities, concerns, and recommendations for the application of LLMs to support health equity.³

Opportunities

Individual health

Behavior change and disease knowledge represent key components of health equity initiatives for chronic disease management which may be augmented with LLMs. One example for addressing diabetes disparities is the Diabetes Self-Management Education and Support program (DSME/S), which provides support for behavior change.⁴ Despite the evidence for DMSE/S, low participation and limited personalization have hampered effectiveness, especially among marginalized communities. Promising work using text messaging lays the foundation for LLMs to impact chronic disease disparities.⁵ LLMs could offer patients personalized and interactive versions of these programs by adapting the health information to meet a patient’s literacy level and preferred language. For example, a patient admitted to the hospital with newly diagnosed diabetes would use an LLM-powered chatbot to learn about their disease and be better prepared to speak with a diabetes educator. The chatbot would use recommended health literacy techniques for education, like the teach-back method, to explore patient understanding.⁶ Further, once the patient leaves the hospital, the chatbot would provide personalized lifestyle recommendations and educational content adapted to patients’ culture and preferences.

Patients from marginalized communities, especially those with lower health and digital literacy, find it hard to access their data. The 21st Century Cures Act, which gives patients increased access to their health information, underscores the need to support patients in engaging with their data. LLMs can summarize complex medical text in patient-friendly language. For example, using an LLM-powered tool, a patient being discharged from the hospital can learn from their notes. These tools could also serve as digital assistants that help patients navigate the healthcare system. Rather than navigating a complex user interface that requires high digital literacy, patients will use LLMs to ask: “What medications were changed?,” “What are my follow up appointments?,” or “Can you help me reschedule my appointment?” This changes patients’ interaction with current information, like after-visit summaries, which often contains complex language.

Family and organizational health

LLMs could also strengthen the patient-clinician relationship and empower patients and care partners from marginalized communities. By providing support before, during, and after visits, LLM-powered tools can help patients prepare for visits by creating relevant questions about their care in the appropriate literacy level.⁷ During the visit, LLM-powered tools could help patients remember the generated questions, note clinician answers for later review, or understand consent forms. For example, Mirza et al⁸ used LLMs to improve readability of surgical consent forms from a reading level of 12.6 to 6.7. After the visit, LLMs could answer patient questions, help capture patient reported outcomes and preferences, which would inform medical decision-making by integrating the patient voice in care.^9,10

Community health

LLMs have the potential to allow healthcare systems to connect patients with community resources to address social needs. This requires screening patients, referring to resources, and ensuring that patients can access these resources. This process can be onerous for patients and clinical teams alike. LLMs can augment the work of community health workers and existing programs by not only screening patients, but also facilitating the application process for resources. For example, an LLM-powered tool could screen patients for food insecurity, then help patients find their nearest food pantry or help patients navigate the application for food benefits. Additionally, there is already an example of using LLMs to extract social determinants of health (SDOH) information from free text notes, which could facilitate the identification of patients in need of resources.¹¹

Patients from marginalized communities experience worse healthcare access, which may be mitigated with LLM-based applications. Patients can use LLMs to write their secure messages and help clinical teams respond to messages in multiple languages and literacy levels. This is already being deployed and could be expanded to support non-English languages.¹² This application would not only contribute to linguistically adapted care, but also address the challenge of message volume.

Population health

LLMs can facilitate care across the healthcare system and increase operational efficiency to help bridge quality gaps, especially in healthcare settings with limited resources. Patients from marginalized populations are more likely to have multiple sites of care, requiring clinicians to review information from multiple sources. LLMs can provide clinicians with summarized clinical courses, which can minimize repetitive testing and progress patient care.

For healthcare leaders, LLMs can enhance data analysis and recognize quality gaps affecting marginalized populations. For example, leaders could use natural language to query data (eg, electronic health record data, social determinants of health data) to identify disparities in chronic disease quality metrics.¹¹ This can augment existing data analytics infrastructure.

At the bedside, clinical decision support powered by LLMs is poised to provide clinicians with easier access to evidence-based care, especially in areas lacking access to specialists. Further, administrative hurdles to prescribing have already been described as targets for LLMs.¹³ This can help address challenges in access to medications and help achieve “pharmacoequity.”¹⁴

Concerns

Like other AI tools, the potential benefits of LLMs come with the risk of exacerbating health disparities. First, LLMs are trained on data that contain historical and selection biases which could propagate discrimination.¹⁰ There are known biases in the internet data used to train LLMs, and there is stigmatizing language in clinical notes.^15–17 LLMs trained on data influenced by commercial entities could preferentially recommend specific treatments. Additionally, languages that are not well-represented in the training data may not be well supported for machine translation. These translations have the potential to cause communication errors and create safety concerns.¹⁸ Second, organizations serving marginalized communities have lagged in their uptake of technologies, like decision support in electronic health records, and may lack the technical infrastructure to leverage LLMs. Consequently, these organizations will be less likely to contribute patient data to further train LLMs, thus worsening performance for patients receiving care at underserved organizations. Third, patients may be hesitant or prefer to maintain their personal connections with their care teams, rather than rely on LLM-powered tools. These changes in the patient-clinician relationship could be particularly harmful for patients living in areas with low healthcare access where health systems may be more likely to implement LLMs. Fourth, mistrust in the healthcare system could be worsened by a lack of privacy and transparency in the development and use of LLMs. Fifth, affordability poses a barrier to patients and healthcare systems. While some consumer-facing LLMs are freely available, paid options could pose a barrier to access for marginalized populations. Sixth, if LLM use is expanded, marginalized populations may be predominantly impacted by their negative effects if the models underperform for their use cases or due to challenges presented by hallucinations that patients with low health and digital literacy may not be able to identify.

Recommendations

We highlight recommendations grounded in the NIMHD levels of influence (noted in parentheses) to ensure that LLMs are developed and implemented with a commitment to health equity. Central to all future LLM work is the participation of patients, especially from marginalized communities, in development, implementation, and evaluation. These recommendations add to prior work in AI and digital equity.^19–21

Engineer prompts that support health equity (individual, community, societal)

The inputs or prompts provided to LLMs should be refined to ensure the output is accurate, unbiased, and language, and literacy appropriate. Engineering appropriate prompts will require collaboration between researchers, developers, clinical teams, and patients. For example, by using few-shot prompting informed by health equity principles (eg, culturally tailored care), we may be able to improve the LLM output. Recommendations for dietary recommendations could be tailored to dietary preferences based on a patient’s cultural background. These examples should be derived iteratively through collaboration with communities where the LLMs will be deployed.

Leverage retrieval augmentation and fine tuning to adapt to patient needs (individual, interpersonal, community)

LLM tools can provide personalized information to patients by leveraging data that includes culturally and linguistically diverse health information. Two methods provide the foundation for this application: (1) retrieval augmentation generation, where relevant external sources are retrieved and integrated into the prompt to improve the reliability of LLM output, and (2) fine tuning, where pretrained LLMs are further trained on data for a specific task.^22,23 Cultural adaptation of health information is a nuanced process that requires multidisciplinary input to tailor information to patient’s “norms, beliefs, values, and literacy skills.”^24,25 To culturally adapt LLM output, developers and researchers can use previously developed health materials (eg, patient information sheets, recipes, text message campaigns) as source material for retrieval-augmented LLM tools focused on specific disease management.²⁶ For example, an LLM-based tool can retrieve and incorporate relevant culturally adapted diabetes self-management materials to increase the accuracy and relevance of the LLM output. Healthcare organizations can collaborate with researchers to create trustworthy sources for training LLMs. This could allow the creation of retrieval-augmented chatbots for specific diseases that would enable the patient to look back at trusted source material. Worse LLM performance on less common languages may be mitigated by the creation of smaller task specific LLMs trained on linguistically diverse text.²⁷ Fine tuning LLMs can support further tailoring, though it requires additional training data and technical expertise which may limit its scalability.

Prioritize digital inclusion (individual, community, societal)

Digital inclusion, or the ability for all patients and communities to have a fair and just opportunity to benefit from digital tools, should remain a focus of healthcare systems as well as local, state, and national governments. Technology access disparities may become increasingly relevant as LLM-based tools for health information-seeking and patient communication are released. Healthcare and community organizations should continue efforts to increase access (eg, broadband internet and devices) to digital tools, support digital literacy, and address misinformation. As LLMs become integrated into search engines, digital inclusion efforts should focus on working with patients to understand the limitations of the information presented or how they can change their search terms or prompts to retrieve accurate information. Additionally, without appropriate focus on equitable diffusion, clinics serving patients from marginalized communities may lag in LLM uptake, thus limiting their potential impact. Healthcare systems serving marginalized communities may require additional funding and support for the technical infrastructure required for successful implementation and maintenance of LLM-based technologies.

Develop transparent notification and consent procedures (interpersonal, societal)

Balancing the use of LLMs in healthcare with the potential impact on patient trust requires transparency.²⁸ Organizations should develop easily understandable consent forms to allow patients to understand how their data will be used (ie, will it be used for training data) and offer an opportunity to opt-out. After deployment, organizations should notify patients when LLMs will be used for their care. The most pressing example is the use of LLMs to respond to patient messages. As a first step, organizations may include a notification in the message letting patients know that an LLM has been used to respond to their question. However, for some patients the concept of LLMs or AI may be unfamiliar. Healthcare organizations should include language that is created with the input from patients and communities. The language would note the use of LLMs, but also promotes trust in the response. The use of these transparent notifications can mitigate mistrust in technology and the healthcare system.

Continuous monitoring of LLM-based technologies for biases (community, societal)

As LLM-based technologies are deployed in clinical care, it will be critical to appropriately evaluate for biases. For example, evaluations can determine if LLMs respond differently to clinical questions depending on patient demographics.²⁹ Institutions could use “red teaming” or “jailbreaking,” which tests models for vulnerabilities that might lead to unwanted outputs, to probe for biases.³⁰ These efforts could be informed by health equity work that has identified structural and systemic barriers to care. These evaluations could be included in policies regulating the use of LLMs for care.^31,32

Conclusions

Healthcare systems and vendors alike face the challenges of implementing new technology while emphasizing equity. This will require funding and collaboration between multiple stakeholders to match the drivers of health disparities with the unique abilities of LLMs. While prior technologies have left marginalized patients behind, LLMs have great potential to improve care, only if health equity is focused on from the start.

Author contributions

All authors contributed to the conception, drafting, revising, and final approval of the manuscript.

Funding

J.A.R. receives funding for this research from the National Institute of Minority Health and Health Disparities grant number K23MD016439. The views expressed in this article are those of the authors and do not necessarily reflect the views and policy of the National Institutes of Health.

Conflicts of interest

D.W.B. reports grants and personal fees from EarlySense, personal fees from CDI Negev, equity from ValeraHealth, equity from Clew, equity from MDClone, personal fees and equity from AESOP, personal fees and equity from FeelBetter, personal fees and equity from Guided Clinical Solutions, and grants from IBM Watson Health, outside the submitted work. Additional authors do not have competing interests to declare.

Data availability

No new data were generated or analyzed in support of this research.