Abstract
Artificial intelligence (AI) has the potential to transform healthcare by enhancing quality, improving safety, and increasing value. However, achieving these benefits requires navigating a complex interplay between data quality, interoperability, computability, workflow integration, and governance, as well as the need to create and sustain what have been described as learning health systems. The full promise of AI will only be realized through rigorous data stewardship, the design and demonstration of novel computational methods, cross-disciplinary collaboration, and a commitment to responsible and equitable technology implementation. Substantial work remains to address these challenges and capitalize on these opportunities, which can and should lead to significant improvements in human health and well-being.
Keywords: Biomedical informatics, artificial intelligence, healthcare transformation, healthcare policy, quality, safety
Introduction
It can be argued that healthcare is entering a transformative era, where artificial intelligence (AI) is no longer a speculative idea but instead is becoming an operational reality. For this article and to contextualize the preceding statement, AI can be defined as:
“the capability of computer systems or algorithms to imitate intelligent human behavior”
(Source: Merriam-Webster Dictionary)
Across a spectrum from the use of natural language processing (NLP) to interpret unstructured clinical documentation to the use of predictive modeling of patient trajectories to the application of large language models to synthesize complex and multi-scale knowledge resources, AI offers an unprecedented opportunity to catalyze improvements in the quality, safety, and value of healthcare delivery.1,2 Yet, these opportunities must be balanced with the need to implement AI technologies in a complex ecosystem, where fundamental questions regarding data quality, interoperability, computability, workflow integration, and governance remain. These challenges must be understood and addressed if we are to realize the benefits of AI. To this end, in the remainder of this editorial, brief summaries of such challenges and associated opportunities for innovation and impact are provided.
Data Quality and Fitness for Use
First and foremost, AI models are only as reliable as the data on which they are trained and deployed. A recent systematic review of electronic health record (EHR) data quality assessment (DQA) methods highlights persistent variability and a lack of standardization in evaluating critical dimensions of structured clinical data quality, including completeness, correctness, plausibility, and concordance.3 Despite decades of progress, there remains no broadly adopted DQA framework capable of addressing such concerns in a systematic and reproducible manner. As a result, we continue to see a proliferation of AI systems that amplify data imperfections or produce inaccurate results due to inadequate training or operational data. There remains a critical need for the creation and dissemination of accessible, efficient, and generalizable DQA methods that can render diverse, multi-scale clinical data “fit for use,” particularly in the context of AI applications. Furthermore, the ability of such methods, if developed, to provide rigorous measures of potential sources of bias in datasets would be highly beneficial in terms of informing mitigation plans (e.g., bias cannot be mitigated if it is not first identified).
Unlocking Textual and Imaging Data
A significant proportion of the information necessary to inform clinical quality and safety improvement in healthcare resides in semi-structured or unstructured text, such as clinical notes. Similarly, high-value phenotypic data are often found in images produced by pathology, radiology, and other ancillary sources. Traditional rule-based or statistical natural language processing (NLP) pipelines have proven valuable in extracting computable features from textual assets; however, they often lack scalability and generalizability. Large language models (LLMs) are now demonstrating equivalent or superior performance when compared to more traditional NLP methods when extracting complex phenotypes from various types of clinical notes, while also requiring less time and customization to achieve such results.4,5 Similarly, machine-learning-based approaches to image processing have been demonstrated to facilitate the extraction of computable features from various imaging modalities. However, they too suffer from concerns regarding scalability and generalizability.4,5 Again, the use of emergent generative AI methods (GAI) is beginning to show promise in terms of enabling broader and less computationally intensive approaches to image processing and feature recognition. As these methods mature, they should help to render increasingly large amounts of unstructured and imaging data actionable in a manner that supports and enables downstream analysis, such as the use of AI. In effect, this is an example of how AI tools can facilitate the production of data that is needed for “downstream” analytical approaches (e.g., AI-enabled AI). As such techniques become more common and easy to use, they will allow the creation and use of AI tools that reason across scales of measurement and data in a manner consistent with the intrinsic complexity of human health and disease. Therefore, continued and intensive research is needed to develop, verify, and validate such methods, which are foundational to the future of AI in healthcare.
Translating Data into Evidence
The systematic conversion of “fit for use” and multi-scale clinical data into evidence that can inform care transformation is a persistent and consequential bottleneck.6 Data streams captured across electronic health records, imaging systems, and complementary platforms, as described above, are often ambiguous, biased, or incomplete, reflecting variations in documentation practices, inconsistent coding, and gaps in longitudinal follow-up.3,6 These limitations complicate efforts to generate reliable insights and risk amplifying existing inequities if left unaddressed. At the same time, they create opportunities for innovation. For example, AI methods can be explicitly designed to model uncertainty, reconcile discordant information, and surface patterns that are not readily apparent using more traditional methods. Similarly, computational learning techniques can quantify and mitigate biases that may exist through the analytic pipeline. At the same time, multi-modal data fusion approaches can enable cross-validation of phenotypes derived from structured, semi-structured, and unstructured sources. Further, temporal and trajectory-based data modeling can allow for the detection of subtle disease progression “trajectories” that are difficult to discern through alternative methods such as statistical modeling. These types of tools and techniques can open the door to robust real-world evidence generation, where data derived from learning health systems (as discussed shortly) is translated into evidence in a timely and efficient manner, and then disseminated via mechanisms that are responsive to diverse populations and aligned with emerging value-based care objectives. The challenge to doing so is that we must create computational solutions that can manage data quality and provenance at scale, while also developing and applying sociotechnical frameworks and constructs, such as governance, interoperability standards, and evaluation metrics, that collectively ensure AI-enabled insights are trustworthy, equitable, and actionable. This tension between data imperfection and methodological innovation may represent the most significant risk and the most significant promise for AI-enabled healthcare transformation.
AI and the Learning Health Systems
The tight coordination of AI development and deployment with healthcare IT (HIT) best practices is crucial for realizing the full transformative potential of such intelligent systems in healthcare. As health systems continue to generate unprecedented volumes and varieties of data, as introduced above, the challenge is no longer simply storing or transmitting this data, but instead converting it into models, tools, and insights that are both scientifically rigorous and operationally useful.
Three guiding axioms emerge based on these observations, namely that: 1) we should treat data as a renewable and continually enrichable resource; 2) we must ensure that the translation of algorithms and analytical products into actionable knowledge is a routine and expected outcome of the innovation enterprise; and 3) we must prioritize the rapid, responsible return of AI-driven insights to clinicians, patients, and other stakeholders.7 These principles are part of a broader framework for building what is often referred to as a learning health system, where HIT and analytical capabilities support and enable continuous learning and improvement.8 In this formulation, the learning health system serves as the effector arm of AI capabilities, aiming to enhance the quality, safety, and value of care, as discussed at the outset of this manuscript (Figure 1). Yet, the realization of an AI-enabled learning health system is not without intrinsic challenges. Critical and open questions remain, in addition to those noted above, such as how to align incentives across researchers, clinicians, patients, and policymakers; ensuring transparency and accountability in the design and deployment of technology; and building a workforce capable of bridging the gap between data science and healthcare delivery. However, much like the preceding examples, such challenges present powerful opportunities. By confronting them directly, we can pioneer new methods for equitable and explainable AI, design governance frameworks that balance innovation with safety, and foster collaborations that make continuous learning an operational norm rather than an aspirational goal. In this sense, building an AI-enabled learning health system is both a grand challenge and the culmination of applying AI “at scale” in the health and life science environment. It requires not only technical innovation but also cultural and organizational transformation. If these challenges are embraced as opportunities for discovery and reform, the result can and should be a resilient, adaptive healthcare ecosystem where AI continuously drives improvements in quality, safety, and value.
Figure 1.
The role of AI and data science across the whole learning cycle, from data generation and analysis (afferent processes) to knowledge delivery, evaluation, and impact (efferent processes). This continuous feedback loop connects healthcare operations and biomedical research, ensuring that insights derived from data are transformed into digital interventions, evaluated for effectiveness, and reintegrated to optimize patient care and system performance.
Discussion
Ensuring Safety and Equity in AI Deployment
Significant risks counterbalance the transformative potential of AI in healthcare. Models trained on biased, incomplete, or poorly curated data, as noted above, risk perpetuating existing inequities and even introducing new forms of harm. Insufficient transparency regarding process automation can erode clinician situational awareness, undermining trust and safety at the point of care. Furthermore, universal guidelines for evaluating the quality of AI inputs and outputs are underdeveloped, resulting in significant variability in model reliability and reporting across different healthcare systems and use cases. Equity concerns are particularly urgent; structured datasets often overrepresent advantaged populations while undercapturing the lived experiences of individuals from marginalized or underserved communities. This imbalance threatens to amplify disparities in diagnosis, treatment, and outcomes. To avoid these types of pitfalls, AI must be deliberately designed to incorporate socioeconomic, racial, and geographic diversity, with fairness and inclusivity treated as fundamental performance metrics rather than aspirational goals.2,9
Moving From Efficiency Gains to Care Transformation
AI’s contribution to healthcare value has often been framed in terms of incremental efficiency gains, such as reducing administrative burden, optimizing scheduling, or automating routine documentation. While these improvements are essential, they represent only the beginning of AI’s transformative potential. By enabling the real-time synthesis of multimodal data, as introduced here, AI can generate scalable, real-world evidence that supports both individual clinical decisions and population-level health management strategies. This capability opens the door to care redesign, shifting from reactive treatment models to proactive approaches that anticipate disease progression, optimize resource allocation, and tailor interventions to the unique needs of individual patients. When such AI solutions are integrated thoughtfully into value-based care frameworks, AI holds the promise of not only lowering costs but also improving outcomes, enhancing patient experience, and advancing the broader goals of precision health and population health management.1,2,4,10
Challenges and Future Directions
Recognizing the significant benefits of AI in healthcare will necessitate addressing several inherent challenges. Standards for interoperability remain fragmented, limiting the ability to share and reuse models across health systems. Regulatory pathways for adaptive AI, tools that evolve as they encounter new data, are still in their infancy, raising challenging questions about evaluation, accountability, and approval processes. Equally pressing is the need to prepare the healthcare workforce, equipping clinicians, administrators, and data scientists with the knowledge and skills necessary to use AI responsibly and effectively. Transparency must also become a non-negotiable feature of AI systems, ensuring that models can be interrogated, explained, and trusted. Looking forward, several priorities stand out. Federated learning provides a pathway to build robust and generalizable models while maintaining data privacy. Emerging “AI assurance utilities” can provide ongoing, independent evaluation of model performance, safety, and bias. Finally, participatory design approaches, engaging clinicians, patients, and communities directly in the development process, can ensure that AI is not only technically sophisticated but also socially aligned, equitable, and sustainable.
Conclusion
In summary, the future of AI in healthcare hinges on our ability to strike a balance between innovation and responsibility, ensuring safety and equity, delivering value that extends beyond efficiency gains, and addressing the technical and sociotechnical challenges that remain. By embracing data as a renewable resource, prioritizing transparency and inclusivity, and embedding AI within a continuously learning health system, we can transform fragmented experiments into sustainable improvements in quality, safety, and value. The path forward requires not only technical breakthroughs but also governance, workforce readiness, and the active engagement of clinicians, patients, and communities. If these elements align, AI will not simply automate aspects of healthcare; it will help reimagine the way health systems learn, adapt, and deliver care for all populations, while ensuring the quality, safety, and value of that care are maintained at both the individual and community levels.
Footnotes
Philip R.O. Payne, PhD, is Director, Institute for Informatics, Data Science, and Biostatistics, the Janet and Bernard Becker Professor, Vice Chancellor for Biomedical Informatics and Data Science, WashU Medicine, and Chief Health AI Officer at BJC Health System and Washington University Medicine, St. Louis, Missouri, USA.
Disclosure: No financial disclosures reported. Artificial intelligence, language models, machine learning, or similar technologies were not used in the conceptualization, study, research, preparation, or writing of this manuscript.
References
- 1.Goldberg CB, Adams L, Blumenthal D, et al. To do no harm—and the most good—with AI in health care. Nejm Ai. 2024;1(3):AIp2400036. doi: 10.1038/s41591-024-02853-7. [DOI] [PubMed] [Google Scholar]
- 2.Matheny ME, Goldsack JC, Saria S, et al. Artificial Intelligence In Health And Health Care: Priorities For Action: Article examines priorities for the uses of artificial intelligence in health and health care. Health Aff (Millwood) :10–1377. doi: 10.1377/hlthaff.2024.01003. Published online 2025. [DOI] [PubMed] [Google Scholar]
- 3.Lewis AE, Weiskopf N, Abrams ZB, et al. Electronic health record data quality assessment and tools: a systematic review. J Am Med Inform Assoc. 2023;30(10):1730–1740. doi: 10.1093/jamia/ocad120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kohane IS. Injecting artificial intelligence into medicine. Nejm Ai. 2024;1(1):AIe2300197. [Google Scholar]
- 5.Lee P, Goldberg C, Kohane I. The AI Revolution in Medicine: GPT-4 and Beyond. Pearson; 2023. [Google Scholar]
- 6.Bastarache L, Brown JS, Cimino JJ, et al. Developing real-world evidence from real-world data: Transforming raw data into analytical datasets. Learn Health Syst. 2022;6(1):e10293. doi: 10.1002/lrh2.10293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Payne PR, Wilcox AB, Embi PJ, Longhurst CA. Better together: integrating biomedical informatics and healthcare IT operations to create a learning health system during the COVID-19 pandemic. Learn Health Syst. 2022;6(2):e10309. doi: 10.1002/lrh2.10309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Friedman C, Rubin J, Brown J, et al. Toward a science of learning systems: a research agenda for the high-functioning Learning Health System. J Am Med Inform Assoc. 2015;22(1):43–50. doi: 10.1136/amiajnl-2014-002977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Longhurst CA, Singh K, Chopra A, Atreja A, Brownstein JS. A call for artificial intelligence implementation science centers to evaluate clinical effectiveness. NEJM AI. 2024;1(8):AIp2400223. [Google Scholar]
- 10.Yadav GS, Longhurst CA. Will AI Make the Electronic Health Record More Efficient for Clinicians? NEJM AI. 2025;2(3):AIe2500020. [Google Scholar]


