Increasing acceptance of AI‐generated digital twins through clinical trial applications

Abstract

Today's approach to medicine requires extensive trial and error to determine the proper treatment path for each patient. While many fields have benefited from technological breakthroughs in computer science, such as artificial intelligence (AI), the task of developing effective treatments is actually getting slower and more costly. With the increased availability of rich historical datasets from previous clinical trials and real‐world data sources, one can leverage AI models to create holistic forecasts of future health outcomes for an individual patient in the form of an AI‐generated digital twin. This could support the rapid evaluation of intervention strategies in silico and could eventually be implemented in clinical practice to make personalized medicine a reality. In this work, we focus on uses for AI‐generated digital twins of clinical trial participants and contend that the regulatory outlook for this technology within drug development makes it an ideal setting for the safe application of AI‐generated digital twins in healthcare. With continued research and growing regulatory acceptance, this path will serve to increase trust in this technology and provide momentum for the widespread adoption of AI‐generated digital twins in clinical practice.

INTRODUCTION

Our healthcare system is fraught with complex problems that prevent patients from receiving quality medical care. Rising costs and inequitable access to health care are significant barriers to improving health outcomes globally. ¹ Furthermore, current methods for evaluating the effects of different treatments on individuals are inefficient. ² The concept of a digital twin (DT) of a patient has recently emerged in healthcare as a powerful tool to address these issues. In principle, DTs can support clinical decision‐making by forecasting the effects of different treatments (including standard‐of‐care or placebo), either individually or in aggregate over a population. Recent advancements in AI that leverage the power of big data are making accurate and useful DT technology a reality. The ability to virtually examine the safety and efficacy of new treatments would greatly expedite evidence generation. However, there is still significant reluctance from both providers and patients to adopt AI practices in medicine, ³ and a systematic regulatory approach to address concerns about using DTs for personalized medicine would serve to enhance trust in these technologies. ⁴

Here, we offer a definition of AI‐generated DTs of patients and suggest that the acceptance of this technology in clinical trials, a highly regulated field, will lead to broader acknowledgment of their utility in clinical practice. First, we define and explore the current applications of AI‐generated DTs in healthcare to anticipate their future impact as a transformative technology for personalized medicine. Next, we consider how this technology can improve the efficiency of clinical trials. Finally, we discuss the limitations of adopting these technologies in healthcare.

DEFINING AI‐GENERATED DTS IN HEALTHCARE

While DT technology has been embraced by the fields of engineering and manufacturing for decades, ⁵ the concept of employing DTs in healthcare has only recently gained traction. In general, a DT is defined as “a mathematical model with an updating mechanism that generates data which are indistinguishable from its physical counterpart.” ⁵ This model of a real‐world entity permits virtual interactions that support forecasting, experimentation, and simulation involving that entity. In cardiology, DTs of the heart have been used to examine in silico organ‐drug interactions (Dassault's Living Heart) and plan cardiac procedures (FEops HEARTguide). ⁶

Although mechanistic (or physics‐based) models have been successful in creating DTs of a single organ system, creating DTs of complex biological organisms requires a model that can handle both the large number of variables and the intricate relationships among them. In oncology, digital twins of cancer patients have demonstrated success in predicting chemotherapy responses and optimizing cancer therapy. ⁶ Unlike DTs of organ systems, a DT of a patient is a virtual representation of that individual, typically built from physical theory, multimodal patient data, population data, and/or real‐time updates on patient and environmental variables. ⁷

Since mechanistic models rely on static or predefined parameters to directly simulate the granular physics of the system they are modeling, well‐calibrated generative AI models are more suited to produce accurate forecasts of how a patient's condition is likely to develop without requiring nearly as many structural assumptions. Therefore, an AI‐generated DT of a patient is a virtual representation of that patient, created by modern AI methods applied to large clinical datasets, from which predicted trajectories that are statistically indistinguishable from the patient's real data can be generated. Technologies that enable the rapid and precise prototyping of intervention strategies in a holistic manner could eliminate a large portion of the guesswork in both drug development and clinical practice and fast‐track medical innovation (Figure 1).

FIGURE 1.

The path to personalized medicine through AI innovation and regulatory acceptance. The path begins with the first publication of artificial neural networks, ²⁵ which paved the way for the modern AI revolution. The development of deep learning techniques continued until the mid‐1970s when interest in connectionist models and federal funding for AI technologies began to wane. After a period of stalled innovation known as the “AI Winter,” improvements in the performance of computers allowed for a resurgence of AI in the 2000s. Now, in 2024, we are at a crossroads of implementing advanced AI technologies in healthcare. The downward trajectory shows a path towards a costly and inefficient healthcare system if data privacy barriers, delayed regulatory involvement, and public fear and distrust in these technologies are not overcome. On the other hand, regulatory acceptance and improved data access would allow for broader applications of AI‐based methods in healthcare so that the path of AI innovation can lead us toward equitable and accessible personalized medicine.

In their infancy, AI systems relied on logic‐based symbolic reasoning to draw conclusions from explicit rules that were hard‐coded into the algorithm, forming the basis of predictive modeling in medicine (e.g., linear regression). The advent of deep learning techniques allowed connectionist models to make more precise predictions through increased exposure to vast amounts of data the way that the human brain does (e.g., artificial neural networks). ⁸ Now, we can build larger, general models that learn context and meaning from neural networks (e.g., transformers). An example is Med‐PaLM, a large language model that can accurately answer medical questions by training on a massive dataset of medical text and code. ⁹ Of note, scientific literature can also be used alongside patient data to train AI‐generated DT models for predicting treatment effects, both in clinical trials and in clinical practice.

AI‐generated DTs have already gained traction in numerous healthcare applications due to their predictive power and ability to supplement, rather than replace, clinical judgment. In the future, these algorithms could also facilitate personalized medicine by identifying patterns in multidimensional clinical data, helping physicians create tailored diagnoses and treatment plans. ⁷ Building trust is crucial for AI model adoption so that healthcare practitioners and patients alike can benefit from these tools. This could be achieved through the highly regulated space of clinical trials in drug development, where we can incorporate the outputs of AI‐generated DTs in a manner that is consistent with current regulatory requirements.

THE UTILITY OF AI‐GENERATED DTS IN CLINICAL TRIALS

As an analogy, consider the advancements in aerospace design with the creation of DTs of aircraft. Instead of undertaking the lengthy and costly process of collecting data on a design's effectiveness and safety after full deployment, aerospace engineers can use accurate computer simulations to study and perfect designs before real‐world implementation. Similarly, the lengthy and costly nature of investigational therapies can be streamlined by adopting AI‐generated DTs in drug development. Rather than undertaking a series of experiments to assess the properties of a new therapy, users of AI‐generated DTs in clinical trials can simultaneously investigate and address a broad set of questions with a single, comprehensive, and accurate model.

While studies have shown success in utilizing deep learning and neural networks to predict clinical trial outcomes, ¹⁰ AI‐generated DTs offer the unique capability of also providing accurate variance estimates for predicted patient outcomes, informing clinical trial design. Fisher et al. ¹¹ demonstrated an unsupervised machine learning (ML) model that simulated comprehensive Alzheimer's disease trajectories indistinguishable from actual patient data by logistic regression. DT technology has been demonstrated by others as having immense potential for improving the efficiency of clinical trials. ¹²

Alam et al. ¹³ comprehensively detail an updated model architecture used to generate DTs across multiple disease indications. In a clinical trial setting, baseline data are collected from each participant prior to randomization and input into a disease‐specific AI model trained on historical control data from previous clinical trials or disease registries. The AI model can then be used to create a probabilistic forecast of how that participant's clinical measurements (e.g., symptoms, biomarkers, etc.) would evolve over the course of the trial if they were randomly assigned to the control group.

These predicted outcomes can be used in a variety of ways to optimize clinical trial design and analysis, such as informing subgroup discovery and interim analysis. Additionally, a prognostic score can be derived from a participant's AI‐generated DT. Because prognostic baseline covariates increase the statistical power of a clinical trial, these derived prognostic scores are among the most predictive factors that one can obtain. This method, known as prognostic covariate adjustment (PROCOVA), is a special case of the analysis of covariance (ANCOVA) method; PROCOVA has been qualified by the European Medicines Agency (EMA), ¹⁴ and aligns with existing guidance from the U.S. Food and Drug Administration (FDA). ¹⁵ Because PROCOVA can enable reductions in variance, it allows for the use of smaller control groups to achieve the same study power. This decreases the amount of time and resources required to bring effective treatments to patients while better aligning the trial design with the participants' wishes (Figure 2).

FIGURE 2.

Comparison of a traditional phase 3 randomized controlled trial (RCT) versus a Phase 3 RCT with AI‐generated digital twins. On the left, a traditional phase 3 RCT shows a 1:1 randomization ratio with an equal number of participants in both the active treatment arm and the control arm of the study. On the right, AI‐generated digital twins are created for each participant, regardless of study arm assignment. The information from these digital twins can be used in the design and analysis of a clinical trial to decrease sample size while maintaining power or to increase power while maintaining sample size.

Covariate adjustment with a composite “score” that is the output of a prognostic model is not a novel concept. The Framingham cardiovascular risk score, for example, was developed using Cox and logistic regression models on a large community‐based cohort to create a single, highly predictive covariate for cardiovascular outcomes. ¹⁶ Approaches that combine traditional statistical methods with ML have shown promise in identifying risk factors, ¹⁷ forecasting health expenditure, ¹⁸ and detecting life‐threatening infections in infants. ¹⁹ These hybrid models can often improve predictive accuracy while preserving interpretability and/or explainability, which has been a focus of regulators in their development of standards for trustworthy AI. ²⁰ , ²¹

To satisfy the rigor of regulatory requirements, many developers have begun to provide explainability metrics as a way to assess model robustness. SHapley Additive exPlanations (SHAP) are a popular method to quantify how each model feature contributes to individual predictions, allowing us to better understand the reasoning behind a model's output. ¹³ Regulators may also request Local Interpretable Model‐Agnostic Explanations (LIME), a technique that approximates black box models with a local, interpretable model to explain each individual prediction. ²¹ Because missing data are so prevalent in many healthcare applications, it is also important to test how robust a model's performance is to missing inputs. Input sensitivity can measure how much a given performance metric changes when a certain feature is masked from the patient's baseline data. ¹³

AI‐generated DTs can help to ameliorate many of the troubles that plague clinical trials, such as their exorbitant cost, lengthy process of experimentation, and high failure rate. ² The use of AI‐generated DTs can lower the costs of drug development by reducing sample size, bringing down enrollment times, and shortening the total duration of clinical trials, benefiting patients, providers, government, and industry alike. ¹³ Within the realm of enhancing clinical trial design, AI‐generated DTs also have the potential to assist with selecting inclusion and exclusion criteria and establishing more sensitive outcome measures. Ideally, the design of a trial should incorporate rich prognostic measurements that better capture individual differences rather than the current method of optimizing measures for group‐level statistical analysis. Because AI models are capable of accounting for these individual differences, future trials that incorporate this technology could benefit from less rigid inclusion and exclusion criteria, as well as be able to accommodate more complex standards of care for comparison. In this sense, AI‐powered clinical trials would allow for improved generalizability of the study's results to the target population.

For clinical trials in rare diseases or pediatric cases, it can be challenging to enroll enough participants to achieve sufficient study power. Bayesian methods can be employed to improve the precision of treatment effect inferences in small trials. Historical data can be encoded as “prior information” and integrated with adjustments using Bayes' Theorem. This process produces updated beliefs about the treatment effect, known as the “posterior distribution.” By leveraging additional information from historical control data, the sample size of the small trial can be augmented, increasing the power for treatment effect inferences, particularly in combination with AI‐generated digital twins. ²²

In addition to enhancing the design of clinical trials, AI‐generated DTs can also improve the efficiency of drug development by reducing the burden for patients to participate in trials. ²³ Patients may also be hesitant to participate in clinical trials due to the possibility of being assigned to a control group rather than receiving the active study drug. ² By reducing the number of participants needed to take the placebo, AI‐generated DTs allow for trial design to better accommodate patients' desires to receive the experimental treatment upon enrolling in a clinical trial. With this, trial sponsors can improve participant recruitment and accrual to prioritize the needs of patients seeking experimental therapies. The continued development of this technology, coupled with increasing regulatory support and advanced trial designs, will help to open doors for the evolution of AI systems from drug development to the clinic.

LIMITATIONS OF AI‐GENERATED DTS

Although the concept of DTs in engineering has become broadly recognized, creating a virtual replica of an inanimate object is not the same as creating a DT of a living organism. The limitations involved with creating DTs of humans can be grouped into four main categories: complexity and variability of the physical system, data availability and standardization, prognostic value and validation, and ethical/privacy concerns.

Complexity and variability

One of the foremost limitations of creating DTs of patients is the inherent complexity of human biology. Unlike mechanical systems with well‐defined components and standard designs, each patient has a unique health profile that is influenced by numerous factors, including genetics, environment, lifestyle, and random variations. At a molecular level, cells communicate through intricate networks of signaling pathways and metabolic processes which differ significantly due to genetic variation and external influences. At a physiological level, numerous systems interact in highly dynamic and nonlinear ways that are difficult to simulate and predict under different scenarios, particularly in the case of mechanistic DTs. In order to build high‐quality AI models for personalized forecasting of disease progression, novel architectural approaches must be developed to model the evolution of many clinical characteristics. As the standard of care evolves due to medical advances and the availability of new therapies over time, so do the societal factors that influence disease prevalence and progression. Building models that maintain their safety and utility requires robust procedures for validation, updating with new data, and application. These procedures must consider the context of use to control risks and minimize the impact of variability in performance, especially in complex settings. Addressing these challenges is key to unlocking the full potential of DT technology in personalized medicine and improving patient outcomes.

Data availability and standardization

Moreover, these models must be able to integrate health data from disparate sources. Electronic health records (EHRs) contain an abundance of information acquired at the point of care, but are collected in a wide variety of formats and often include poorly labeled diseases and conditions. Existing AI models are constrained by both the availability of these data and by the algorithm's ability to learn from these records due to the inherent heterogeneity of disease processes. ¹¹ On the other hand, clinical trial data and other sources of peer‐reviewed evidence are considered to be of (comparably) high quality, but typically only constitute a few hundred patients and may not represent the broader disease population.

Building high‐caliber datasets that capture global patient variability in a fair and equitable way requires aggregating and harmonizing data from diverse sets of regionally limited studies. This is an especially important challenge when data collection conditions change due to data drift, changing standard of care, or the introduction of novel measurements. Limitations in the ability to build consistent, representative datasets require the development of algorithms better equipped for generalization.

Transfer learning can be applied to enhance the standardization of EHR data. This approach allows for pre‐trained models from related domains to be adapted to the healthcare sector. For example, in natural language processing (NLP), knowledge from NLP models can be leveraged to extract information from clinical notes. ²⁴ Similarly, features learned from general image datasets can be transferred to medical imaging tasks through image recognition to improve the accuracy and efficiency of data processing.

Prognostic value and validation

The interaction of various biological systems and external factors makes it difficult to create accurate predictions of the distribution of likely health outcomes while simultaneously having the precision required for applications. It is also difficult to predict how patients will react to new treatments, as the outcomes can vary widely between individuals. These problems are exacerbated in scenarios where the amount of prognostic information is limited, or certain data are unavailable due to systematic conditions in study design.

AI models must be able to make predictions while also accurately modeling the distributions of likely outcomes. They must be able to capture the complex interactions between all of the variables in play, function in the presence of missing data, and handle data of all types (categorical, ordinal, continuous, etc.). The exponential growth in research and monetary investment in AI architectures has largely been focused on consumer applications with data that is fundamentally quite different from healthcare data. Focused effort must be made to develop AI models for human health to create effective, useful applications.

Validation of models is a cornerstone to developing applications of AI. Careful consideration of the context of use is critical to determine how to characterize model performance and evaluate risks. Robust validation also requires sufficiently relevant data to demonstrate the utility of models and value for applications.

Ethical and privacy concerns

Patient data are sensitive and protected by laws to ensure both privacy and consent for its use in clinical research. Huang et al. (2022) created a process‐oriented map of the various ethical concerns related to incorporating DTs for personalized health, grouping the concerns into data collection, data management, data analysis, and information use. The authors highlight the need for cybersecurity investments to protect data integrity with DTs, and careful validation procedures to avoid discriminatory health outcomes from biased algorithms or training datasets. ⁴ Quantifying model performance and testing the model on independent datasets allows users to assess the accuracy of prognostic data, and continuously monitoring deployed models reinforces the reliability of these AI models.

CONCLUSION

AI‐generated DT solutions can already be implemented in a regulatory‐aligned and low‐risk manner in clinical trials. Our continued work on AI‐generated DTs has demonstrated the ability of these models to improve the efficiency of randomized clinical trials. ¹¹ , ¹³ By developing effective and technologically feasible AI solutions for healthcare‐specific problems, and working to increase regulatory and public trust in novel AI technologies, we can help to prepare the medical field for the transformation of these powerful tools from infrequent research applications to routine clinical practice.

FUNDING INFORMATION

No funding was received for this work.

CONFLICT OF INTEREST STATEMENT

All authors are equity‐holding employees of Unlearn.AI, Inc., a company that creates digital twin generators to forecast patient outcomes. The authors declared no other competing interests for this work.

Vidovszky AA, Fisher CK, Loukianov AD, et al. Increasing acceptance of AI‐generated digital twins through clinical trial applications. Clin Transl Sci. 2024;17:e13897. doi: 10.1111/cts.13897