Abstract
INTRODUCTION
Machine learning models leverage baseline data to create artificial intelligence (AI)–generated digital twins (DTs)—individualized predictions of each participant's clinical outcomes if they had received placebo. Incorporating DTs may increase statistical power or reduce required sample sizes in Phase 2 or 3 trials, and therefore improve efficiency in Alzheimer's disease (AD) trials. Here we demonstrate these properties using data from an AD Phase 2 clinical trial (AWARE, NCT02880956).
METHODS
A conditional restricted Boltzmann machine (CRBM) model was trained on historical clinical trials and observational data from 6736 unique subjects after data harmonization to generate DTs of participants from the AWARE trial. The AWARE trial enrolled 453 subjects with mild cognitive impairment (MCI) or mild AD. DTs were assessed as prognostic covariates to evaluate gains in variance and sample size reduction.
RESULTS
Positive partial correlation coefficients were found between DTs and change score from baseline in key cognitive assessments ranging from 0.30 to 0.39 at Week 96 in the AWARE trial. These correlations were consistent with validation results from three independent trials, which ranged from 0.30 to 0.46. Total residual variance was reduced by ~9% to 15% with DTs. While maintaining statistical power, DTs could reduce total sample size by ~9% to 15%, and control arm sample size by 17% to 26% in future AD trials.
DISCUSSION
Efficiency was improved in AD clinical trials using machine learning models to generate prognostic DTs by including them in statistical analysis modeling. This methodology aligns with regulatory guidance and represents an application of machine learning models suitable for the analysis of pivotal trial data. Validated DTs have the potential to improve clinical development efficiency in AD and in other neurological indications.
Highlights
Digital twins (DTs) were generated by artificial intelligence (AI) models trained on historical datasets.
Use of digital twin (DT) as a covariate in the analysis model can reduce treatment effect variability.
By coupling DT with the analysis model, trial sample size can be reduced.
DT technology was accepted by the U.S. Food and Drug Administration and European Medicines Agency for applications in clinical trials.
Keywords: AI‐generated digital twins, Alzheimer's disease, artificial intelligence, clinical trial efficiency
1. INTRODUCTION
Alzheimer's disease (AD) trials are often slow and costly, driven by long follow‐up, high screen failure rate, and large sample size requirements. 1 , 2 Participants at similar stages of the disease can progress at different rates, and high variability in common clinical outcomes is a major driver of the large trial size and long recruitment periods. Another significant challenge in drug development, in particular AD, is that the results from smaller, underpowered Phase 2 trials often fail to be replicated in larger Phase 3 trials. Having a better estimate of the treatment effect from Phase 2 could reduce the failure rate in Phase 3.
Few tools exist to reduce trial sample sizes without compromising validity. One approach is to develop novel endpoints or use surrogate biomarkers to replace traditional endpoints; however, in many neurological disease indications, developing novel endpoints is both challenging and time‐consuming. Another approach is to reduce the size of the study population to limit heterogeneity. This creates challenges in determining real‐world benefits and applicability of trial results to broad patient populations. Finally, some statistical methodology has been developed to use historical data to address this problem, such as pooling trial data or borrowing historical data to increase effective sample size, Bayesian methods that integrate the distribution of historical data as a prior distribution, 4 and machine learning‐enabled external control arms. 5 , 6 However, each of these techniques carries the risk of bias due to the potential for consequential differences in unobserved variables. As a result, their applications to clinical trials remain limited, especially in AD.
In exploring new means of accelerating drug development, artificial intelligence (AI) has the potential to address challenges in clinical development. Traditional statistical methods can struggle to account for many variables and subtle interactions that influence disease progression, whereas AI could identify complex non‐linear relationships. AI can improve enrollment by matching patients to suitable clinical trials, informing eligibility criteria with electronic health record data, and even analyzing multimodal imaging markers to select ideal participants. 7 The growing interest in using this transformative technology to accelerate drug development has also spurred the advancement of trial participants’ digital twins (DTs), which can provide data‐driven insights that enable drug discovery, support in silico trial design, and even serve as external control arms. 8 , 9 Using DTs to simulate detailed patient characteristics and forecast disease progression in AD can help address some challenges involved in developing effective treatments. 10 , 11
A DT of an individual can be defined as a “computational model that describes the evolution of their health over time.” 12 For our purposes, the generative AI model that creates each participant's DT is a system of neural networks trained on data from historical clinical trials and observational studies. The model forecasts placebo outcomes from each participant's baseline information, providing a comprehensive, longitudinal, and computationally generated clinical record. This forecast of a participant's future placebo outcomes can be incorporated in the statistical analysis model as a covariate in a clinical trial to reduce statistical uncertainty, increase power, and/or potentially reduce the enrollment duration, reduce the size of the placebo group, and other applications. 13 The combination of statistics and AI is intended to provide ways to leverage historical datasets and models of disease progression to produce evidence in clinical trials that is statistically robust and clinically meaningful. This approach, known as prognostic covariate adjustment (PROCOVA), received a positive qualification opinion by the European Medicines Agency (EMA) as a novel methodology for medicines development in September 2022. 14 Similarly, PROCOVA‐Mixed‐Effects Model for Repeated Measures (MMRM) is an extension of PROCOVA for longitudinal continuous outcomes. 15 Here we evaluate this methodology using data from the AWARE trial, a Phase 2 clinical study of tilavonemab in patients with early AD. 16 , 17
2. METHODS
DTs of participants from the AWARE trial were generated using a conditional restricted Boltzmann machine (CRBM), 10 an unsupervised machine learning model with probabilistic neural networks that were pre‐trained and validated prior to its application to the trial. We assessed the properties and characteristics of these DTs, the output of the trained CRBM, including their comparability and potential to reduce sample size in PROCOVA models.
RESEARCH IN CONTEXT
Systematic review: Machine learning models leverage historical baseline data to create artificial intelligence (AI)–generated digital twins (DTs)—individualized predictions of each participant's clinical outcomes if they had received placebo. Incorporating DTs may increase statistical power or reduce sample sizes in Phase 2 or 3 trials, and therefore improve efficiency in Alzheimer's disease (AD) trials.
Interpretation: Positive partial correlation coefficients were found between DTs and change score from baseline in key cognitive assessments ranging from 0.30 to 0.39 at Week 96 in the AWARE trial. These correlations can be translated to total residual variance reduction from ~9% to 15% with DTs. While maintaining statistical power, DTs could reduce total sample size by ~9% to 15%, and control arm sample size by 17% to 26% in future AD trials.
Future directions: Validated DTs have the potential to improve clinical development efficiency in AD and in other neurological indications.
2.1. AWARE trial dataset
The AWARE trial (NCT02880956) was a multicenter, multiple‐dose, randomized, double‐blind, placebo‐controlled Phase 2 study to evaluate tilavonemab in treating early AD. The primary and secondary efficacy endpoints were the change from baseline to Week 96 in the Clinical Dementia Rating‐Sum of Boxes (CDR‐SB) score, 14‐item Alzheimer's Disease Assessment Scale–Cognitive subscale (ADAS‐Cog 14) total score, Mini‐Mental State Examination (MMSE) total score, and Functional Activities Questionnaire (FAQ) total score. A total of 453 patients were randomized in 1:1:1:1 ratio to receive one of three doses of tilavonemab or placebo, of whom 337 and 116 were assigned to tilavonemab groups and placebo group, respectively. Baseline data from the AWARE trial were used as input for DT generation for each participant in the trial. Baseline variables included as input in the CRBM are presented in Supplementary Information A.
2.2. Historical training dataset
To develop the AD prognostic model, a harmonized dataset was created by combining data from two databases of longitudinal clinical data on AD. One source is the C‐Path Online Data Repository for Alzheimer's Disease (CODR‐AD) provided by the Critical Path for Alzheimer's Disease (CPAD) 18 consortium, which consists of the control arms of 29 clinical trials mostly in mild to moderate AD with >7000 trial participants. The other source is from the Alzheimer's Disease Neuroimaging Initiative (ADNI) 19 database, a collection of four long‐running observational studies enrolling 1728 participants across the AD disease spectrum beginning in 2004, focused primarily on participants that were cognitively normal or considered to have mild cognitive impairment (MCI). The resulting integrated dataset had 6736 participants (Table 1), covering 66 background and longitudinal variables, such as demographics, genetics, clinical severity, component level cognitive and functional measures, and laboratory measurements. Both clinical and observational studies in MCI (~25%) and AD patients were included. Variables included in the integrated dataset used to train the generative AI model are presented in Supplementary Information B.
TABLE 1.
Baseline characteristics of trial participants whose historical data were included in the integrated dataset for model training.
| Range | |||||
|---|---|---|---|---|---|
| Baseline characteristics (n a ) | Mean value (SDb) | Minimum | Maximum | ||
| Age, years (6734) | 73.7 (8.3) | 45 | 90 | ||
| Education, years (1405) | 15.7 (2.9) | 4 | 20 | ||
| Baseline MMSE score (3677) | 22.2 (4.6) | 0 | 30 | ||
| Baseline CDR‐SB score (1454) | 2.7 (2.2) | 0 | 16 | ||
| Baseline ADAS‐Cog 11 score (6665) | 20.4 (10.6) | 1 | 68 | ||
| Baseline FAQ score (1398) | 6.2 (7.2) | 0 | 30 | ||
| Sex (6736) | Female | Male | |||
| 3525 | 3211 | ||||
| Race (6728) | White | Black | Asian | Other | Not specified a |
| 6236 | 152 | 214 | 126 | 8 | |
| Ethnicity (3170) | Hispanic/Latino | Non‐Hispanic/Latino | Not specified a | ||
| 227 | 2943 | 3566 | |||
| APOE ε4 genotype (2234) | Positive | Negative | Not specified a | ||
| 1265 | 969 | 4502 | |||
| Amyloid status (1405) | Positive | Negative | Not specified a | ||
| 1013 | 392 | 5331 | |||
Abbreviations: ADAS‐Cog 11: 11‐item Alzheimer's Disease Assessment Scale–Cognitive subscale; APOE, apolipoprotein E gene; CDR‐SB, Clinical Dementia Rating–Sum of Boxes; FAQ: Functional Activities Questionnaire; MMSE, Mini‐Mental State Examination; SD, standard deviation.
n represents the available sample size for the given variable. Participants who were recorded as N/A or unknown are not included in n and are presented as “Not specified” in the last column of the table.
Continuous variables are reported as mean and SD, and the minimum and maximum recorded values are also provided.
2.3. Conditional restricted Boltzmann machine (CRBM)
Boltzmann machines are machine learning algorithms for modeling relationships among data. Conditional restricted Boltzmann machines (or CRBMs) extend Boltzmann machines such that multiple time points are learned simultaneously, containing visible units for multiple timepoints with a single hidden layer. The static units (baseline predictors) are used only once over all time points, and the model learns the complete joint distribution between all adjacent time points simultaneously, creating a prediction that is conditional only on baseline values. Further details on the machine learning algorithm can be found in Fisher et al. 10
Because the historical training data consists only of placebo‐arm data, the model's outputs are predictions of disease progression in AD, given their specific baseline features. The model does not predict treatment effects or treatment trajectories. For modeling, the historical datasets were harmonized by aligning variable definitions, standardizing measurement scales, and mapping outcomes to a common format across sources. The harmonized historical data were split into training, tuning, and test datasets, consisting of 50%, 20%, and 30% of the data, respectively. The goal of the modeling approach was to produce predictions of disease progression that accurately predict the probability distribution of actual participants.
2.4. CRBM tuning
The tuning dataset was used to compute metrics such as mean squared errors and discrimination metrics that judge model performance, and optimal hyperparameters were selected. Finally, the CRBM was then retrained using these optimal hyperparameters on both the training and tuning components of the dataset. After retraining the CRBM on the combined training/tuning dataset, the baseline characteristics from study participants included in the test dataset were used to quantify model performance.
We further performed validation of the CRBM on three independent clinical trials which targeted the mild‐to‐moderate AD population. They consist of a study on docosahexaenoic acid (DHA; NCT00440050), 20 a clinical trial on the effects of valproate (NCT00071721), 21 and a clinical trial on the effects of resveratrol (NCT01504854). 22
2.5. Digital twins
Predictions of disease progression from the CRBM are known as DTs, as they represent a copy of a trial participant. The predictions are generated from the unique baseline measurements of all trial participants. These predictions are probabilistic distributions, and the aggregate of the predictions represents the unique DT of each participant in a clinical trial. An overview of the DT creation and evaluation steps is presented in Figure 1. Predictions are made for all common cognitive and function measures in an AD trial, including ADAS‐Cog, MMSE, Alzheimer's Disease Cooperative Study–Activities of Daily Living (ADCS–ADL), FAQ, CDR‐SB, and Neuropsychiatric Inventory (NPI); biomarkers including total tau, phosphorylated tau‐181 (p‐tau181), amyloid beta (Aβ42); and common laboratory and vial measurements.
FIGURE 1.

Overview of data harmonization, training, inference, and application of DTs. Data harmonization is the first step, and involves data cleaning, harmonizing units, and unifying studies. The second step is to train and tune the model using the harmonized data. After the model has been validated, the locked model can be used for inference, by generating DTs of each participant in a clinical trial. DTs are probability distributions, and therefore patient trajectory samples are aggregated or reduced for each participant's digital twin (DT) through its moments (mean, variance, etc.). DT, digital twin.
2.6. Application of DTs
Although DTs represent predictions of disease progression, an adaptor such as a statistical method drives the application of DTs in clinical development suitable in regulatory settings. In fact, the use of DTs directly as an external control arm or as a counterfactual does not control bias and generally limits the potential of DTs. We propose to use DTs as a prognostic factor in the trial analysis such that randomization is preserved and suitable for registrational purposes. One such application is covariate adjustment, or more specifically, prognostic covariate adjustment (or PROCOVA), a linear model that adjusts for the prognostic DT prediction as a covariate. 23 This class of statistical models has robust properties including strict Type I error control, and is commonly used and accepted throughout regulatory agencies. PROCOVA inherits these properties and estimates of treatment effect are unbiased, with the addition of improved precision if the DT is prognostic. The overall goal for this application of DTs is to generate a covariate that is more prognostic than any single or set of baseline covariates that are commonly adjusted for. This methodology is fully pre‐specifiable if the model has been validated and locked before generation of DTs, making it suitable as the primary analysis for pivotal trials. 14
Following the guidance of the U.S. Food and Drug Administration's (FDA) on covariate adjustment: “Sponsors can adjust for baseline covariates in the analyses of efficacy endpoints in randomized clinical trials. Doing so will generally reduce the variability of estimation of treatment effects and thus lead to narrower confidence intervals and more powerful hypothesis testing” (p. 3). 24 “In a trial that uses covariate adjustment, the sample size and power calculations can be based on adjusted or unadjusted methods. The latter will often lead to a more conservative sample size” (p. 3). 24 PROCOVA may be suitable for use to support trials with smaller sample sizes and increased power compared to unadjusted methods. In linear statistical models, it is known that the Pearson correlation coefficient between the dependent outcome variable and independent predictor variable is directly associated with the variability of the outcome explained by the predictor variable. Therefore, the most important metric for this context of use is the Pearson correlation coefficient between the outcome variable and prognostic DT prediction. Assuming a balanced trial with equal variances between treatment groups, the variance reduction compared to an unadjusted model is the Pearson correlation coefficient squared. In addition, the bias or prediction error of the DT does not have impact on the unbiasedness of the estimation of the treatment effect when it is used as a covariate in the statistical analysis model, and therefore the bias of the DT is not an important metric for this methodology.
In our implementation, adjusting for the prognostic score captures key baseline demographics and clinical scales, and adjusts for differences in these baseline features between treatment groups in cases where randomization fails to balance in finite sample size. However, unmeasured or underrepresented features may not be accounted for fully. Trialists may therefore consider enrolling controls enriched for such features that are not available to the model, to ensure balance and completeness in the control population.
2.7. Evaluation of DTs in the AWARE trial
DT predictions were generated for each participant in the AWARE trial by applying baseline data to the CRBM model. We evaluated the potential of DTs to reduce sample size or increase statistical power through a retrospective comparison of multiple analysis models using the AWARE trial, including longitudinal mixed‐effects models for repeated measures (MMRM), and single timepoint analysis of covariance (ANCOVA) models. In these models we adjusted for the prespecified participant's baseline score or their DTs prognostic score as a covariate. The complete procedure for machine learning model and training is displayed in Figure 1.
3. RESULTS
3.1. Validation performance
We evaluated the performance of the CRBM by comparing the prediction error and Pearson correlations in the held‐out test dataset and in placebo participants from three independent clinical trials datasets. Evaluation was conducted on the control arms only, as the CRBM models were trained to represent disease progression under placebo. Across each endpoint, Pearson correlation coefficients were generally consistent across the three datasets (Table 2). Notably, the three independent trial datasets generally showed higher correlations compared to the held‐out portion of the training data, particularly for CDR‐SB and MMSE. This indicates that overfitting was not present. There was moderate prediction error (bias) in each of the three evaluations; however, in this application of DTs, only the correlation between the predicted DT score and observed outcomes influences variance reduction. The prediction bias does not bias the estimation of treatment effect in this application.
TABLE 2.
Pearson correlations (r) and prediction error for validation of conditional restricted Boltzmann machine (CRBM).
| Endpoint |
1Held out portion of training data r (95% CI) Mean prediction error |
2Docosahexaonoic acid trial dataset (DHA) r (95% CI) Mean prediction error |
3Valproate trial dataset (VALID) r (95% CI) Mean prediction error |
4Resveratrol trial dataset (RN) r (95% CI) Mean prediction error |
|---|---|---|---|---|
| CDR‐SB score | 0.319 (0.060–0.542) 0.49 | 0.306 (0.137–0.457) 1.00 | 0.394 (0.228–0.538) 1.01 | 0.461 (0.204–0.659) 0.17 |
| 5ADAS‐Cog total score | 0.355 (0.152–0.528) 1.32 | 0.322 (0.147–0.477) 2.78 | 0.267 (0.052–0.458) 3.24 | 0.358 (0.103–0.569) 2.85 |
| MMSE total score | 0.397 (0.218–0.550) 0.557 | 0.566 (0.433–0.675) –1.20 | 0.509 (0.359–0.663) 0.22 | 0.662 (0.470–0.794) 1.61 |
Abbreviations: ADAS‐Cog 11: 11‐item Alzheimer's Disease Assessment Scale–Cognitive subscale; CDR‐SB, Clinical Dementia Rating–Sum of Boxes; MMSE, Mini‐Mental State Examination.
1Validation of model performance on a subset (30%) of the training data that were held out for the validation purposes and three completely independent clinical trials that were separate from the training data. The duration of DHA2 (NCT00440050, N = 164), VALID3 (NCT00071721, N = 160), and RN4 (NCT01504854, N = 60) was 18 months, 24 months, and 12 months, respectively. 5DHA used ADAS‐Cog 11, VALID used ADAS‐Cog 13, and RN used ADAS‐Cog 14.
3.2. Evaluation on the AWARE trial
We evaluated DT predictions against observed outcome change scores in the AWARE study using Pearson and partial correlations. The correlation coefficients between DT prognostic scores and observed outcome scores ranged from 0.2 to 0.5 in the AWARE study (Supplementary Information C). The partial correlation coefficients at Week 96 for four scales were positive and ranged from 0.3 to 0.39. These partial correlation coefficients can be used as an estimation of correlation coefficients between DT prognostic scores and observed outcome scores, supporting the designs of future similar clinical trials.
We compared observed outcomes measured by CDR‐SB, ADAS‐Cog 14, MMSE, and FAQ in the AWARE trial with corresponding DT scores generated from the CRBM model with AWARE trial baseline variables as input parameters (Supplementary Information D). Overall, DT scores showed higher mean scores for CDR‐SB and FAQ, whereas ADAS‐Cog 14 and MMSE mean DT scores were lower than observed mean outcomes scores. DT prognostics scores consistently showed smaller variability, as demonstrated by smaller standard deviations (SDs) of DT prognostic scores than observed outcomes across all scale measures. We also assessed mean differences between DTs and corresponding observed outcome scores using paired t‐tests; significant differences were observed across outcomes (Supplementary Information D).
3.3. Comparison of variance reduction
Figure 2 shows the reduction in total residual variance when DT prognostic scores are included as covariates in ANCOVA. These variance reductions were used for sample size calculations. Compared to an analysis of variance (ANOVA) model without any covariate adjustment, the ANCOVA model adjusting for baseline score as a covariate reduces <5% of total residual variance. However, total residual variance reduction by DT prognostic score for all treatment groups combined ranges from ~9% to 15% in PROCOVA models. The magnitude of total residual variance reduction depends on clinical assessment scales and is the highest for the MMSE total score.
FIGURE 2.

Percent of total residual variance reduction among different methods comparing to ANOVA model without any covariate adjustment. “Adj Baseline” represents the ANCOVA model using baseline score as a covariate. “DT” represents the ANCOVA model using DT PROCOVA. Participants in placebo group and one‐dose group are included in the dose arm displayed. “All Groups” in x‐axis represents participants from three doses groups and placebo group. ADAS‐Cog, Alzheimer's Disease Assessment Scale–Cognitive subscale; ANCOVA, analysis of covariance; CDR‐SB, Clinical Dementia Rating–Sum of Boxes; DT, digital twin; FAQ, Functional Activities Questionnaire; MMSE, Mini‐ Mental State Examination; PROCOVA, prognostic score as a covariate.
We assessed DT‐based variance reduction in both ANCOVA with last observation carried forward (LOCF) and MMRM at Week 96 (Supplementary Information E). In general, the magnitudes of variance reductions for treatment effect estimates by DT prognostic score are comparable between the ANCOVA model with LOCF and the MMRM model. Variance reduction for treatment effect estimates ranges from ~6% to 19% across scales and dose groups in the AWARE trial. Within each scale, variance reduction is not affected by treatment group in the AWARE trial and the range of variance reduction reflects finite sample variability. Overall, variance reduction is consistent between all PROCOVA models, suggesting that DTs are robust to the choice of analysis methods.
3.4. Estimation of sample size reduction
Including DT prognostic scores in an ANCOVA model reduces total residual variance, leading to the benefit of a reduced sample size needed for a trial. Using AWARE data, we estimated the total sample sizes required to achieve 90% power for various assumed treatment effects (25%–35% slowing vs placebo at Week 96). The results comparison for CDR‐SB, ADAS‐Cog 14, MMSE, and FAQ are displayed graphically in Figure 3. Compared to an ANOVA model without any covariate adjustment, total sample size savings were significantly higher for PROCOVA models with DT prognostic score as a covariate in the model. For CDR‐SB, ~13% total sample size savings—or 23% control only sample size—savings could be achieved by DTs. This was consistent with the rule of thumb approximation for control arm sample size reduction (Supplementary Information F) based on the observed partial correlation coefficient of 0.36 and the observed residual variance reduction of 13%. Notably, the 23% sample size savings in the control arm for the DT ANCOVA model (PROCOVA) versus the ANOVA model was much higher than the <5% sample size savings observed for the ANCOVA model (baseline score adjustment) versus ANOVA without adjustment (Figure 3). Table 3 presents residual variance reduction and sample size savings at Week 96 for CDR‐SB score change at Week 96 using ANCOVA or PROCOVA analysis with the AWARE trial data.
FIGURE 3.

Sample size reduction with DT method comparing to an ANOVA model without any covariate adjustment from completers population in the AWARE study. “Adj Baseline”: ANCOVA model adjusting baseline score as a covariate. “DT”: ANCOVA model adjusting DT PROCOVA. Results for all dose groups combined (300 mg, 1000 mg, and 2000 mg dose groups) are presented in this figure. ANCOVA, analysis of covariance; DT, digital twin; PROCOVA, prognostic score as a covariate.
TABLE 3.
Sample size savings by digital twins in prognostic covariate adjustment (PROCOVA) analysis of CDR‐SB score change from baseline to Week 96.
| Residual variance | Overall sample size | Percent reduction | Control arm | Percent reduction | |
|---|---|---|---|---|---|
| 300 mg | |||||
| Adj Baseline * | 6.98 | 938 | 469 | ||
| DT ** | 5.79 | 780 | 16.8% | 334 | 28.8% |
| 1000 mg | |||||
| Adj Baseline | 7.44 | 1000 | 500 | ||
| DT | 6.78 | 912 | 8.8% | 418 | 16.4% |
| 2000 mg | |||||
| Adj Baseline | 7.44 | 1000 | 500 | ||
| DT | 6.67 | 896 | 10.4% | 406 | 18.8% |
Note: Assumed a 25% less decline on CDR‐SB score changes in the treatment group compared to the placebo group for 90% power at one‐sided 0.025 level. Placebo decline was based on observed Week 96 data in the AWARE study. Due to overall variance reduction from DT in the model, the total sample size required for the PROCOVA model is −9% to 17% less compared to the ANCOVA model with baseline score adjustment. If sample size savings are focused only on the control arm, the percentage savings range from −16% to 29%.
Abbreviations: ANCOVA, analysis of covariance; CDR‐SB, Clinical Dementia Rating–Sum of Boxes; DT, digital twin.
ANCOVA model adjusting baseline score as a covariate.
PROCOVA model adjusting DT prognostic score as a covariate.
4. DISCUSSIONS
This work validated the application of AI‐generated DTs to decrease variability in trial outcomes compared to standard methods, while maintaining the robustness of evidence through Type I error rate control and unbiased treatment effect estimates. As a part of model validation, we evaluated DT predictions on a held‐out dataset and in the datasets of three independent clinical trials. DTs showed consistent correlation with observed outcomes across multiple scales with no signs of overfitting. These results support the model's ability to generalize to future trials and reinforce its suitability for prospective applications. A re‐analysis of the AWARE trial data incorporating DT prognostic scores as covariates resulted in reductions of total residual variance and reduced the required overall sample size between 9% and 15%, based on partial correlation coefficients. The use of DTs is in the context of a randomized controlled trial, as DTs do not replace the control arm. This approach demonstrates the value of utilizing AI in ways consistent with regulatory guidance to benefit clinical development.
Several existing methods attempt to utilize historical data to conduct clinical trials more efficiently. However, to our knowledge, DT methodology is the first that produces evidence suitable for registrational studies. This is because it uses the framework of covariate adjustment, which guarantees unbiased treatment effect estimates and Type I error rate control—two essential properties for pivotal studies from a regulatory perspective. Unlike synthetic control methods used in single‐arm or open‐label trials, our approach relies on randomization and preserves unbiasedness and Type I error control. Covariate adjustment is standard in randomized control trial (RCT) analyses, with the difference in this application being that the covariate used is the output of an AI machine learning model that was trained on historical datasets independent from the trial used in the application. This offers the potential for the AI model to leverage more data than a fixed set of standard covariates to provide stronger prognostic information and hence more precise treatment effect estimates.
Limitations in available datasets—such as missing features, underrepresentation of certain populations, or historical biases—may affect DT predictions. In addition, generalization of the model across subpopulations, endpoints, biomarkers, and other factors is critically important but difficult to accomplish. Because clinical trials often investigate many of these at once, a single prognostic model may not be suitable for everything. Finally, the prognostic score may be more difficult to interpret than a standard baseline covariate, although interpretability/explainability tools such as Shapley values can be useful. The CRBM is not a black box model, and the most important features for prediction can be extracted and pre‐specified.
The methodological framework of developing DT prognostic models and using them in trial analyses can be applied to other disease areas. AD is a particularly suitable first case because of the availability of large historical trial datasets and the widespread use of standardized clinical endpoints such as CDR‐SB and ADAS‐Cog. These factors provide both the scale and consistency needed to develop and validate prognostic models. In other disease areas, challenges may arise from smaller trial sizes, greater heterogeneity in outcome measures, or fewer shared baseline covariates across studies.
Because DTs offer predictions of disease progression, they could be utilized throughout clinical development outside of covariate adjustment, often for internal decision‐making that does not require regulatory acceptance. For example, DTs could be used to increase confidence in Go/No‐Go decisions and serve as exploratory external control arms in early‐stage rare disease trials, and to discover key subgroups, predict responders, and refine inclusion/exclusion criteria. Moreover, statistical methodologies such as Bayesian analyses can leverage historical control information with Bayesian PROCOVA for even greater efficiency gains. 25 As the use of AI‐generated DTs gains traction in clinical research, 26 the potential for driving meaningful advancements appears increasingly promising.
CONFLICT OF INTEREST STATEMENT
The authors are employees and/or shareholders of AbbVie and/or Unlearn, which may benefit from the methods described. This represents a potential conflict of interest. The authors have disclosed this relationship in accordance with journal policy.
CONSENT STATEMENT
The study is re‐analysis of the AWARE trial data and consent was not necessary.
Supporting information
Supporting Information
Supporting Information
ACKNOWLEDGMENTS
This work was sponsored and funded by AbbVie. AbbVie and Unlearn collaborated in the analysis and interpretation of the data. AbbVie contributed to the design, research, and interpretation of data, writing, and reviewing, and approved the content. Deli Wang, Hana Florian, Shau‐Yu Lynch, Weining Robieson, and Ole Graff are employees of AbbVie Inc. Run Zhuang, Coco Kusiak, Jessica L. Ross, and Jonathan R. Walsh are employees of Unlearn. Anna Vidovzsky contributed drafting of the manuscript in the early stage when she was employed by Unlearn. All authors may own AbbVie Stock.
Wang D, Florian H, Lynch SY, et al. Using AI‐generated digital twins to boost clinical trial efficiency in Alzheimer's disease. Alzheimer's Dement. 2025;11:e70181. 10.1002/trc2.70181
REFERENCES
- 1. Langbaum JB, Zissimopoulos J, Au R, et al. Recommendations to address key recruitment challenges of Alzheimer's disease clinical trials. Alzheimers Dement. 2023;19:696–707. doi: 10.1002/alz.12737 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Cummings JL, Goldman DP, Simmons‐Stern NR, Ponton E. The costs of developing treatments for Alzheimer's disease: a retrospective exploration. Alzheimers Dement. 2022;18:469–477. doi: 10.1002/alz.12450 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Shi X, Pan Z, Miao W. Data Integration in Causal Inference. John Wiley & Sons, Inc.; 2021; Preprint at https://arxiv.org/abs/2110.01106 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Hill JL, Bayesian nonparametric modeling for causal inference. J Comput Graph Stat. 2011;20:217–240. doi: 10.1198/jcgs.2010.08162 [DOI] [Google Scholar]
- 5. Lee D, Yang S, Dong L, et al. Improving trial generalizability using observational studies. Biometrics. 2021;79(2):1213–1225. doi: 10.1111/biom.13609 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Li X, Miao W, Lu F, Zhou XH. Improving Efficiency of Inference in Clinical Trials with External Control Data. The International Biometric Society; 2021; Preprint at https://arxiv.org/abs/2011.07234v2 [DOI] [PubMed] [Google Scholar]
- 7. Zhang B, Zhang L, Chen Q, et al. Harnessing artificial intelligence to improve clinical trial design. Commun Med. 2023;3:191. doi: 10.1038/s43856-023-00425-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Katsoulakis E, Wang Q, Wu H, et al. Digital twins for health: a scoping review. npj Digit Med. 2024;7:77. doi: 10.1038/s41746-024-01073-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Bordukova M, Makarov N, Rodriguez‐Esteban R, Schmich F, Menden MP. Generative artificial intelligence empowers digital twins in drug discovery and clinical trials. Expert Opin Drug Discov. 2023;19:33–42. doi: 10.1080/17460441.2023.2273839 [DOI] [PubMed] [Google Scholar]
- 10. Fisher CK, Smith AM, Walsh JR, et al. Machine learning for comprehensive forecasting of Alzheimer's disease progression. Sci Rep. 2019;9:13622. doi: 10.1038/s41598-019-49656-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Bertolini D, Loukianov AD, Smith AM. Forecasting progression of mild cognitive impairment and Alzheimer's disease with digital twins. Alzheimers Dement. 2021;17(S9):e054414. doi: 10.1002/alz.054414 [DOI] [Google Scholar]
- 12. Alam N, Basilico J, Bertolini D, et al. Digital twin generators for disease modeling. 2024; Preprint at 10.48550/arXiv.2405.01488 [DOI]
- 13. Schuler A, Walsh D, Hall D, et al. Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score. Int J Biostat. 2021;18:329–356. doi: 10.1515/ijb-2021-0072 [DOI] [PubMed] [Google Scholar]
- 14. European Medicines Agency . Qualification opinion for Prognostic Covariate Adjustment (PROCOVA™) (EMADOC‐1700519818‐907465). CHMP, European Medicines Agency; 2022. Available from: https://www.ema.europa.eu/en/documents/regulatory‐procedural‐guideline/qualification‐opinion‐prognostic‐covariate‐adjustment‐procovatm_en.pdf [Google Scholar]
- 15. Ross JL, Sabbaghi A, Zhuang R, et al. Enhancing longitudinal clinical trial efficiency with digital twins and prognostic covariate‐adjusted mixed models for repeated measures (PROCOVA‐MMRM). 2024; Preprint at https://arxiv.org/abs/2404.17576
- 16. AbbVie . A Study to Evaluate the Efficacy and Safety of ABBV‐8E12 in Participants with Early Alzheimer's Disease. AbbVie. The New England Journal of Medicine; 2022; . Accessed September 27, 2024. ClinicalTrials.gov. https://clinicaltrials.gov/study/NCT02880956 [Google Scholar]
- 17. Florian H, Wang D, Arnold SE, et al. Tilavonemab in early Alzheimer's disease: results from a phase 2, randomized, double‐blind study. Brain. 2023;146:2275–2284. doi: 10.1093/brain/awad024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Critical Path Institute . Critical Path for Alzheimer's Disease (CPAD). Critical Path Institute. Accessed May 16, 2025. https://c‐path.org/program/critical‐path‐for‐alzheimers‐disease/ [Google Scholar]
- 19. Alzheimer's Disease Neuroimaging Initiative (ADNI). Laboratory of Neuro Imaging. Accessed May 16, 2025. https://adni.loni.usc.edu [Google Scholar]
- 20. Quinn JF, Raman R, Thomas RG, et al. Docosahexaenoic acid supplementation and cognitive decline in Alzheimer disease: a randomized trial. JAMA. 2010;304(17):1903–1911. doi: 10.1001/jama.2010.1510 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Tariot PN, Erb R, Podgorski CA, et al. Efficacy and tolerability of divalproex sodium for agitation in dementia: a randomized, double‐blind, placebo‐controlled trial. Am J Geriatr Psychiatry. 2001;9(1):58–66. [PubMed] [Google Scholar]
- 22. Turner RS, Thomas RG, Craft S, et al. A randomized, double‐blind, placebo‐controlled trial of resveratrol for Alzheimer disease. Neurology. 2015;85(16):1383–1391. doi: 10.1212/WNL.0000000000002035 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Li Y, Sabbaghi A, Walsh JR, Fisher CK. Prognostic covariate adjustment for logistic regression in randomized controlled trials. 2024; Preprint at https://arxiv.org/abs/2402.18900
- 24. U.S. Food and Drug Administration . Adjusting for Covariates in Randomized Clinical Trials for Drugs and Biological Products: Guidance for Industry. U.S. Department of Health and Human Services, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, Oncology Center of Excellence; 2023. [Google Scholar]
- 25. Vanderbeek AM, Sabbaghi A, Walsh JR, Fisher CK. Bayesian prognostic covariate adjustment with additive mixture priors. 2024; Preprint at https://arxiv.org/abs/2310.18027
- 26. U.S. Food and Drug Administration . Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products. U.S. Department of Health and Human Services, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research; 2023. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supporting Information
Supporting Information
