Improving Patient Engagement: Is There a Role for Large Language Models?

Since the advent of the electronic health records and their increased adoption, novel ideas regarding asynchronous care and their impact on patient outcomes have emerged. In this editorial, we explore the effect of the utilization of patient portal messaging within radiation oncology, emphasizing both the potential benefits and inherent challenges, particularly focusing on the emerging potential of large language models (LLMs) in improving patient engagement.

In this issue of IJROBP, Alexander et al. analyzed patient portal messaging among radiation oncology patients using data from a single institution. The study included a diverse cohort, assessing both incoming and outgoing messages. They found that portal messaging was associated with better survival: HR was 0.90 for sending messages (95% CI 0.84–0.96; P=0.001) and 0.77 for receiving messages (CI 0.70–0.84; P<0.001), adjusted for socioeconomic, disease, and treatment characteristics. Significant disparities were highlighted, with Black patients less likely to send and be sent messages compared to White patients, and Medicaid patients were less likely to use the portal compared to Medicare patients. Female providers were more likely to send messages than males (OR 1.47; CI 1.34–1.62; P<0.001). These disparities underscore the need for targeted interventions to ensure equitable access to digital healthcare tools.

These results align with other studies that examined the role of patient portal messaging. The use of patient portals has been shown to enhance patient engagement, satisfaction, and perceived health outcomes.¹ Patients tend to appreciate the asynchronous access to their healthcare team, allowing them to ask questions and seek clarification on their own time. This increased engagement may lead to better management of complex diseases and early identification of potential treatment related issues.

While patient portals offer tangible benefits, significant challenges remain. Access to technology is a major barrier for patients from low socioeconomic backgrounds or those with limited technological proficiency.² Among others, Non-English speakers face significant challenges navigating these platforms, which can exacerbate existing healthcare disparities.³

Additionally, the increased use of patient portals can lead to higher workloads for healthcare providers.⁴ The time spent on patient portal messages often goes uncompensated and has been found to associated with increased physician burnout in multiple studies.^5–7 This issue is particularly pronounced among female providers, who tend to spend more time on patient communications, potentially impacting their work-life balance and job satisfaction.⁸

Scalable Engagement: The Potential Role of Large Language Models

To address these challenges and expand the potential of patient portals to improve engagement without increasing physician burden, EHR vendors have begun integrating large language models (LLMs) into patient portal systems as a potential solution. LLMs can draft empathetic and informative responses to patient inquiries, thereby reducing the burden on healthcare providers.^9,10 Potential applications of these models in patient portals include assisting with triaging messages, providing initial drafts, and tailoring responses to meet specific patient needs. This is inherently important in resource-limited healthcare settings.

LLMs have the potential to help draft empathetic, personalized responses that physicians can build on instead of starting from scratch. This has the potential to reduce the workload while generating longer, more consistent, and valuable responses. While time-savings are yet to be fully realized, novel LLM and Natural Language Processing (NLP) tools hold great potential in triaging, summarizing, and drafting. A study by Chen et al.¹¹ noted that LLM drafts generally provided acceptable responses and posed minimal risk of harm, and included more educational content than responses written completely manually. With future iterations, the hope is that these models will be able to generate high-quality, personalized drafts more quickly, thereby increasing patient satisfaction, reducing physician burnout, and potentially improving outcomes. While this alone could help increase adoption, another potential application for LLMs is in quick language translation, which could increase the accessibility and utilization of patient portals for non-English speakers. However, extensive testing and clinical benchmarking is necessary to ensure accuracy and reliability in medical translations.

However, integrating LLMs into patient portal messaging systems represents a first step in a paradigm shift of artificial intelligence-informed healthcare, and carries known and unknown risks. One major concern is the potential for biases in the training data propagating through the models, leading to biased output.¹² For example, if the training data predominantly represents certain demographic groups or has spurious and/or biased associations between these groups and medical facts, the model may generate responses that are less appropriate for underrepresented populations.^13,14 Additionally, since these populations use portals less frequently, their data are less likely to be represented in future training data.

One significant issue with using LLMs is “hallucinations,” where the model generates incorrect or nonsensical information. This is especially dangerous in healthcare, and particularly in oncology, where inaccuracies can have serious consequences. Ensuring that LLM-generated responses are accurate and reliable requires rigorous validation and oversight by healthcare professionals, who are often already pressed for time. Moreover, as LLMs improve and trust in these tools grows, there is a risk of over-reliance on them. This could lead to automation bias and anchoring, with potentially unknown downstream effects. Automation bias occurs when users trust automated systems too much, potentially overlooking errors. Anchoring refers to the human tendency to rely heavily on the first piece of information encountered (the “anchor”) when making decisions. This has already been shown by Chen et al. and merits future study to investigate the impact and optimal mitigation strategies.¹¹

Additionally, integrating LLMs into patient portals raises important questions about privacy and security.¹⁵ LLMs require access to patient data to generate contextually appropriate responses, necessitating robust data protection measures to prevent unauthorized access while ensuring patient confidentiality.

Recent studies of clinical deployment have shown mixed results regarding the time-saving potential of LLMs in healthcare communication. Tai-Seale et al. found that LLMs might reduce cognitive burden but did not save time.¹⁶ Similarly, Garcia et al. noted no change in response time but reductions in cognitive burden and burnout scores.⁹ While these early studies suggest some benefit in reducing cognitive load, it remains unclear if these tools will mitigate burnout, emphasizing the need for further clinical validation. Future research should also explore patients’ perceptions and outcomes of LLM-augmented communication, and how patients feel about healthcare teams using LLMs, especially given some systems charge for portal messages. Awareness of LLM-augmented responses might lead patients to turn to public, general-purpose LLMs, which might be more accessible but do not have the necessary oversight.

Accessible LLMs as a Parallel to Patient Engagement

With the widespread availability of LLM applications and ease of access, patients may eventually realize the benefit of having an application that is willing to engage in discussions related to healthcare and provide tailored advice.¹⁷ These models can provide empathic and personalized answers. Just as patients often turn to Google searches, patients might find it easier and potentially less expensive to ask their friendly LLM about their care instead of asking their care teams. However, it is important to note that LLMs, at present, are not designed to provide reliable clinical content and answers might not be factually correct, especially without clinical nuance., and could relay misinformation present on the internet text that LLMs are pre-trained on. LLMs often provide answers in an authoritative fashion and can mix correct and incorrect information, making it difficult to detect errors.¹⁸ However, instead of discouraging their use entirely, we owe it to our patients to educate them on practical best practices of prompting and fact-checking output. Tips might include how to utilize LLMs to generate meaningful questions to ask their healthcare providers and information summarization, among others.

The utilization of patient portals in radiation oncology offers significant opportunities to enhance patient engagement and thereby improve survival outcomes. However, to fully realize these benefits, it is essential to address the aforementioned challenges related to access, provider workload, and the integration of advanced technologies. Stakeholders must collaborate to develop policies that support equitable access to digital healthcare tools and provide adequate compensation for the additional work associated with patient portal communications.

Technology as a Social Determinant of Health

Underserved communities often lack access to reliable Wi-Fi, computers, and cell phones, and are less likely to be offered patient portals.² If AI-assisted responses provide a net benefit, these populations are less likely to receive them, widening the healthcare gap. On the flipside, over-reliance on inferior AI in under-resourced settings can also increase harm and reduce human engagement. Therefore, investments in AI should be paralleled with investments in the infrastructure and technology that enable the most vulnerable patients to realize the potential benefits of LLMs.

Conclusion

Novel technologies like integrating LLMs into patient portals present a promising avenue to improve efficiency and patient engagement but must be approached with caution. Ensuring these models are accurate, unbiased, and used responsibly is crucial. This is an exciting time for digital healthcare, and the potential to leverage technology for improved patient outcomes is immense. Continued research and thoughtful implementation of these tools will be key to navigating the complexities of modern healthcare and ensuring the benefits are realized across all patient populations.

Data Availability Statement for this Work:

N/A