Disease Modeling and External Model Evaluation Through Clinical Data Sharing Platform for HR+/HER2− Breast Cancer

ABSTRACT

Predictive models for disease progression are valuable for clinical trial design and interpretation; however, suitable data are needed for the development of such models. This study aimed to develop a Tumor Growth Inhibition‐Overall Survival (TGI‐OS) model for hormone receptor‐positive (HR+)/human epidermal growth factor receptor 2 negative (HER2−) breast cancer using clinical trial data available through Vivli, a clinical trial data sharing platform. The CONFIRM study (Phase 3 study comparing fulvestrant 250 vs. 500 mg) was used for model development, and the PALOMA‐3 and SANDPIPER Phase 3 studies (palbociclib and taselisib) were used for external model qualifications. Longitudinal tumor size profiles were first analyzed with the TGI model. The TGI‐OS model, a parametric model linking TGI metrics and baseline predictors of survival outcomes, was then developed using data from the CONFIRM study and showed successful internal qualification, including the prediction of the survival difference between two dose groups. The TGI‐OS model showed large underestimation for the OS for PALOMA‐3; nevertheless, the predicted treatment effect (hazard ratio of OS) was in good agreement with the observation for both studies, suggesting its potential as a tool to support drug development decisions. While integrating shared clinical trial data from multiple sources, facilitated by platforms like Vivli, is crucial for advancing predictive modeling efforts, caution should be exercised when such models are applied for new studies, especially when there are breakthroughs in the treatment landscape.

1. Background

The development of predictive models for disease progression and treatment response is a key aspect of modern clinical development [1, 2, 3]. These models are essential for designing, executing, and interpreting clinical trials, thereby improving our understanding of disease dynamics and the effectiveness of treatments. In oncology, where the dynamics of tumor growth and their response to treatment can be highly complex, reliable predictive models can provide valuable insights that support clinical decision‐making and enhance patient outcomes [4, 5].

One of the main challenges in developing these models is the availability of suitable historical data. Development of disease progression models usually requires data from a large number of patients with sufficient duration of observations, such as data from past Phase 3 clinical trials, in order to fully characterize disease trajectories and variability at multiple levels (within and between subjects, between studies). Several Phase 3 trials might also be required to assess the predictive performance of the models. These data might not be necessarily available during drug development in a single institution, especially when new indications are being evaluated. In recent decades, health authorities and the pharmaceutical industry have recognized the importance of data transparency and collaboration, leading to the implementation of data sharing policies [6, 7]. These policies make individual patient‐level data from clinical trials available, creating unique opportunities for developing disease progression models. Vivli is a platform established in 2018 to accelerate clinical trial data sharing efforts by serving as a platform to connect fragmented clinical trial data [8], and it has successfully accumulated a large amount of trial data from 7000 trials and 3.6 million participants [9].

2. Key Question

In this study, we focus on developing a Tumor Growth Inhibition‐Overall Survival (TGI‐OS) model, one type of disease progression model, for hormone receptor‐positive (HR+)/human epidermal growth factor receptor 2 negative (HER2−) breast cancer. The TGI‐OS model aims to link tumor growth inhibition metrics, such as growth rate (Kg), and prognostic factors to overall survival outcomes [10, 11, 12, 13]. Data suitable for the TGI‐OS model in this indication is particularly scarce because of the relatively long survival time of patients compared to other settings, necessitating longer trial duration to collect sufficient data that allow the linkage of tumor dynamics trajectory to survival outcomes. By leveraging past clinical trial data made available through the Vivli platform, we aimed to overcome this challenge.

Among the clinical trial data on the Vivli platform, two studies were identified as appropriate for our study aim and were approved for sharing by the data contributors: the CONFIRM [NCT00099437] and PALOMA‐3 [NCT01942135] studies. CONFIRM is a Phase 3 study comparing two doses of fulvestrant (250 vs. 500 mg) for HR+ breast cancer patients who experienced progression after prior endocrine therapy, demonstrating the better treatment outcome at the 500 mg dose [14, 15]. PALOMA‐3 is a Phase 3 study evaluating the benefit of combining palbociclib and fulvestrant compared to fulvestrant alone in HR+/HER2− breast cancer patients who had progression or relapse during previous endocrine therapy. While the study showed the benefit of adding palbociclib for progression‐free survival (PFS) [16, 17], it did not show a statistically significant benefit for overall survival [18]. In addition to those studies on the Vivli platform, we utilized internal data from the SANDPIPER study [NCT02340221], which evaluated the benefit of combining taselisib and fulvestrant in HR+/HER2− breast cancer patients with disease recurrence or progression during or after aromatase inhibitor treatment. The study showed a statistically significant benefit of combining taselisib and fulvestrant for PFS; however, the magnitude of clinical benefit was considered modest [19]. Individual patient‐level data from these studies were used for model development (CONFIRM) or qualification (PALOMA‐3 and SANDPIPER).

3. Methods

3.1. Data

Data from three clinical trials were utilized as described above. The CONFIRM study [NCT00099437] [14, 15], which is the oldest among the three studies in scope, was used for the TGI‐OS model development, thus mirroring a real clinical development scenario where the previous Phase 3 trial data was used for model development. The PALOMA‐3 [NCT01942135] [16, 18] and SANDPIPER [NCT02340221] studies were used for external qualification of the model performance.

In this study, we only analyzed patients who had measurable disease at baseline, which was approximately 70%–80% of all the patients whose data were made available in the Vivli platform (Figure S1). For the PALOMA‐3 study, longitudinal sum of the longest tumor diameters (SLD) information was only available in the analysis dataset from the interim PFS datacut [17], and the SLD data from this data cut was used for TGI model development. For the TGI‐OS model evaluation, survival data from the OS data cut was used [18], that is, OS prediction based on the tumor data available at the time of the interim PFS datacut was performed. For the CONFIRM and SANDPIPER study, SLD and OS data were obtained from the same data cut (OS datacut for CONFIRM [15], primary datacut for SANDPIPER [19]). For SANDPIPER, all subjects were included for the TGI modeling, while only the primary efficacy analysis population (subjects with PIK3CA‐mutant tumors, approximately 80% of the total enrolled subjects) were included for the TGI‐OS modeling.

3.2. TGI Model

Longitudinal profiles of the tumor size (sum of the longest tumor diameters (SLD), Figure S2) were analyzed, separately for each study, using non‐linear mixed effects modeling in R with nlme package. The bi‐exponential model [20], as defined below, was used for the structural model, as prior studies have shown that the tumor growth rate constant (KG) estimated from this model is associated with patient survival [10, 11, 12, 13].

$$\mathit{TS}\left(t\right)=\left\{\begin{array}{c}T{S}_0·\\ T{S}_0·\end{array}\right.\begin{array}{c}{e}^{K_{G^{·t}}}\\ \left(e^{-K_{S^{·t}}}+e^{K_{G^{·t}}}-1\right),\end{array}\begin{array}{c}\text{if}t<0\\ \text{if}t\ge 0\end{array}$$

Between‐subject variability was incorporated as log‐normal distributions for KS (shrinkage rate constant), KG (growth rate constant), and baseline tumor size. Fixed effects for KG and KS were separately estimated between treatment arms. The individual post hoc estimates of TGI metrics including Time to Growth (TTG), LnKS (natural log of shrinkage rate constant), LnKG (natural log of growth rate constant), and the tumor size ratio (depth of response) at week 12 or 24 (TR12 and TR24) after the treatment initiation were calculated.

3.3. TGI‐OS Model

Survival models for overall survival (OS) were developed using data from CONFIRM as follows. First, the impact of individual baseline prognostic factors and TGI metrics on OS was explored using univariate Cox regression analysis in R, with the intention to screen relevant prognostic factors prior to parametric modeling, as previously performed [12, 13]. The prognostic factors explored were Age (≥ 65 or < 65 years old), Visceral Involvement, Sensitivity to prior endocrine therapy, 2nd line treatment, Last therapy was anti‐estrogen, Progesterone receptor status, Baseline sum of the longest diameters (mm), as described in Table 1.

TABLE 1.

Demographics of subjects included in the current study.

Trial	CONFIRM		PALOMA‐3		SANDPIPER
Arm	Fluv 250 mg	Fluv 500 mg	Palbo + Fulv	Placebo + Fulv	Taselisib + Fulv	Placebo + Fulv
	254	234	269	138	241	124
Age (years)
≥ 65	105 (41%)	102 (44%)	66 (25%)	37 (27%)	79 (33%)	42 (34%)
< 65	149 (59%)	132 (56%)	203 (75%)	101 (73%)	162 (67%)	82 (0%)
Race
White	248 (98%)	222 (95%)	192 (71%)	99 (72%)	148 (61%)	81 (65%)
Asian	—	—	58 (22%)	29 (21%)	52 (22%)	25 (20%)
Black	—	—	12 (4.5%)	8 (5.8%)	1 (0.4%)	1 (0.8%)
Other/NA	6 (2.4%)	12 (5.1%)	7 (2.6%)	2 (1.4%)	40 (17%)	17 (14%)
Visceral involvement
Yes	176 (69%)	182 (78%)	204 (76%)	104 (75%)	180 (75%)	92 (74%)
No	78 (31%)	52 (22%)	65 (24%)	34 (25%)	61 (25%)	32 (26%)
Sensitivity to prior endocrine therapy a
Yes	83 (33%)	84 (36%)	211 (78%)	106 (77%)	172 (71%)	85 (69%)
No	171 (67%)	150 (64%)	58 (22%)	32 (23%)	69 (29%)	39 (31%)
Last therapy was anti‐estrogen
Yes	147 (58%)	140 (60%)	50 (19%)	20 (14%)	—	—
Progesterone receptor status
Positive	183 (72%)	162 (69%)	189 (70%)	89 (64%)	196 (81%)	99 (80%)
Negative/NA	71 (28%)	72 (31%)	80 (30%)	49 (36%)	45 (19%)	25 (20%)
2nd line b
Yes	133 (52%)	121 (52%)	269 (100%)	138 (100%)	182 (76%)	86 (69%)
No	121 (48%)	113 (48%)	—	—	59 (24%)	38 (31%)
Baseline sum of the longest diameters (mm)	60 (33, 101)	58 (32, 100)	38 (22, 63)	37 (24, 60)	42 (24, 67)	35 (19, 63)
Last tumor size measurement time (days)	116 (80, 329)	162 (82, 449)	38 (22, 63)	37 (24, 60)	168 (109, 278)	114 (67, 227)

Note: Only the subjects with measurable disease at baseline are included in this study as described in Methods. For SANDPIPER, the numbers are for the primary efficacy analysis population (subjects with PIK3CA‐mutant tumors) which was included in the TGI‐OS analysis. Numbers represent n (%) for categorical variables, and Median (1st and 3rd quartile) for continuous variables. —: Not available/applicable or not derived.

^a For CONFIRM, this variable was defined as the sensitivity to the last endocrine therapy.

^b 2nd line was defined as the subjects who experienced endocrine therapy for locally advanced or metastatic disease.

Next, the survival analysis was performed using a multivariate parametric survival modeling approach (accelerated failure time model) using the survreg function in R. The probability density function that best describes the observed survival times was selected among normal, lognormal, Weibull, logistic, log‐logistic, and exponential using the difference in Akaike information criterion (AIC) between the alternative models. A “full” model was built by including all significant covariates screened in the Cox univariate analysis with a significance level of p < 0.05 per the log‐likelihood ratio test where the difference in −2 × log‐likelihood (score) between alternative models follows a χ ² distribution. Then a backward stepwise elimination was carried out. During this phase, fit was assessed on models with one of the covariates removed. If there were covariate(s) for which the removal did not result in worsening of the model fit (at the significance level of p < 0.01), the covariate with the highest p value was removed. This step was repeated until all of the covariates contributed to the model fit performance (i.e., removal of any covariate resulted in worse fit) to reach the final model.

The predictive performance of the model was assessed by posterior predictive checks, where the observations were compared to the 95% prediction intervals of OS distributions and hazard ratios (HRs).

3.4. Software

R version 4.3 or higher was used for data analysis.

4. Results and Discussion

This study leveraged clinical trial data from the Vivli platform to create a model that links tumor growth inhibition metrics to overall survival outcomes, with the aim of providing a valuable tool for clinical trial design and decision‐making.

Table 1 shows the demographics of the subjects included in the study. There were some notable differences between studies: higher representation of Asian subjects in PALOMA‐3 and SANDPIPER, less use of anti‐estrogens as the last therapy in PALOMA‐3, and only 2nd line patients for PALOMA‐3. There were also some differences in variable definitions, such as for the sensitivity to prior endocrine therapies as noted in Table 1. Using the longitudinal tumor size data from these subjects (Figure S2), a simple bi‐exponential TGI model was developed for each study separately, and the longitudinal tumor growth profiles were described well (Figure S3, Table S1).

Next, the TGI‐OS model was developed using data from the CONFIRM study. With the univariable covariate screening, TTG and LnKG were identified as the most significant covariates, followed by clinical baseline covariates including baseline SLD, visceral disease involvement, and age (Table S2). As the TTG and LnKG had similar scores, the subsequent TGI‐OS models were developed with both metrics separately. The backward elimination resulted in the removal of age, and the final model contained LnKG or TTG, baseline SLD, and visceral involvement (Figure 1c, Figure S4c), and the TTG and LnKG models had almost identical log likelihood. The simulation from the developed model with LnKG successfully captured the treatment (dose in this case) response difference (HR) between the two arms (i.e., the slightly longer OS for the higher dose for fulvestrant 500 versus 250 mg) and the survival profiles (Figure 1a,b, Figure S4a,b) without explicitly incorporating the treatment (dose) as the variable in the survival model. The simulated survival profiles by subgroups based on the variables included in the model also matched well with the observation (Figure 1d–f).

FIGURE 1.

TGI‐OS model development and internal qualification with CONFIRM study, using LnKG as the TGI metric. (a, b) Simulated and observed survival and hazard ratios. (c) Model equation and the coefficients See Text S2 for more details of the model. (d–f) Simulated and observed survival for subgroups of subjects based on the covariates in the TGI‐OS model. For the survival profiles, the ribbons represent 95% prediction intervals of simulated survival profiles from 1000 simulations with TGI‐OS model, and the lines represent the observed survival profile estimated with Kaplan–Meier method. For the hazard ratio plot, the histogram represents simulated HR from 1000 simulations, the dashed vertical lines represent 95% prediction intervals of the simulations, and the red vertical line represents the observed hazard ratio estimated with Cox proportional hazard model. LnKG, BLSLD, and VISCINV represent the natural log of tumor growth rate, baseline sum of the longest diameters, and visceral disease involvement, respectively.

As the final step, the developed TGI‐OS model was applied on the two external studies, PALOMA‐3 and SANDPIPER. The developed model with the parameter values and an executable code example is available in Text S1. Simulated survival showed large underprediction of the observed survival for PALOMA‐3, while that for SANDPIPER was consistent with the observation (Figure 2). The reason for such discrepancy is unclear. One possible reason is the use of tumor data from an earlier datacut for PALOMA‐3 (interim PFS). Because of this, tumor data for PALOMA‐3 is much shorter in duration (Table 1, Figure S1) and this could have resulted in a biased estimate of TGI metrics, as can be seen in a faster estimated LnKG population estimates than the other studies (Table S1). Another possibility is a shift in the treatment landscape in the HR+/HER2− BC space between the time of CONFIRM and PALOMA‐3 or SANDPIPER studies. Indeed, a previous study showed that the use of a disease model developed using clinical data from the 2000s failed to capture the survival profiles of a more recent study performed around 2020, indicating the importance of using contemporaneous data in developing such disease models [21]. Furthermore, there are other differences in the patient characteristics between studies, such as PI3K mutation status, prior treatment, and race (proportion of Asian patients).

FIGURE 2.

TGI‐OS model external qualification with (a, c) PALOMA‐3 and (b, d) SANDPIPER study. (a, b) Simulated and observed survival profiles with the LnKG model and (c, d) those with TTG model. For the survival profiles, the ribbons represent 95% prediction intervals of simulated survival profiles from 1000 simulations with TGI‐OS model, and the lines represent the observed survival profile estimated with Kaplan–Meier method. For the hazard ratio plot, the histogram represents simulated HR from 1000 simulations, the dashed vertical lines represent 95% prediction intervals of the simulations, and the red vertical line represents the observed hazard ratio estimated with Cox proportional hazard model.

As the aim of the study was to evaluate the prospective application of the TGI‐OS model based on earlier available data, the patient factors not available in the CONFIRM study were not assessed as prognostic factors. Of note, our goal here is to use those models to support drug development decisions. What we care most about is the prediction of the treatment effect, that is, OS hazard ratio of experimental vs. control arms rather than the prediction of OS that might be impacted by multiple unmeasured differences between studies. The hazard ratio between two arms was captured reasonably well: 0.82 (observed, recalculated in this study for the patients with measurable disease) versus 0.81 (0.66–0.99) (predicted with LnKG model) for PALOMA‐3; 0.89 (observed) versus 1.0 (0.85–1.23) (predicted with LnKG model) for SANDPIPER, respectively (Figure 3). There was some dependency of the simulated HR on the TGI metric of choice for PALOMA‐3. This difference might be attributable to the differences in mechanisms of action between treatments, and further evaluation is warranted. For the SANDPIPER study, some difference between the observed and simulated HR might be attributable to the immature survival data. It is noteworthy that, while PALOMA‐3 showed a strong PFS benefit of palbociclib, the TGI‐OS model prediction based on the tumor data that was available at the time of the PFS data cut accurately predicted a more modest OS HR at the OS data cut. These results indicate the potential utility of the developed model in reliably estimating treatment benefits for future studies, while the poor prediction of survival distributions in PALOMA‐3 warrants further investigations.

FIGURE 3.

Simulated and observed hazard ratios. For the observed hazard ratios, circles and the bars represent the point estimates and 95% confidence intervals estimated with the Cox proportional hazard model. For the simulated hazard ratios, circles and the bars represent the median and 95% prediction intervals from 1000 simulations with the TGI‐OS model. LnKG and TTG model: Simulated HR from the TGI‐OS model using either the natural log of tumor growth rate or time to tumor regrowth as a key TGI covariate. Note that the observed hazard ratios are different from the primary reports from the respective studies, as these were re‐calculated from the available data for patients with tumor size measurement available.

5.

Several limitations are observed from the study, apart from the discrepancy in survival trajectory between the simulated and predicted as described above, which warrants further examination. There was a limitation in the availability of covariates that can potentially explain some of the prognostic differences between the studies, especially because we intentionally utilized data from an old study to develop the model to emulate real‐life scenarios. Also, since we utilized longitudinal tumor size data, we were only able to incorporate information from subjects with measurable disease, which does not represent all the patients in this space. Indeed, the prognosis of patients with bone‐only metastasis is shown to be better than that of the other groups [22], and this can impact the applicability of the TGI‐OS model for studies that enroll a significant proportion of patients with non‐measurable disease; other modeling approaches, for example, multi‐state model, may be able to overcome this challenge.

6. Conclusion

The development of the TGI‐OS model for HR+/HER2− breast cancer indicates the potential of predictive modeling and data sharing in oncology, but at the same time highlights the challenges in extrapolating disease models across studies that have different features such as the inherent treatment landscape shift, patient population differences, and data availability. By utilizing clinical trial data from the Vivli platform, we created a model that links tumor growth inhibition metrics to overall survival outcomes. While survival distributions were diversely predicted across studies, the simulated treatment differences matched reasonably well with the observed, suggesting the potential utility to support clinical drug development. These findings call for further studies to understand the knowledge gaps and enable the extrapolation of a disease model developed in one setting to the other to make it a truly useful and trustworthy clinical drug development tool.

Conflicts of Interest

All authors are current or former employees of Genentech Inc. (a member of the Roche group) and own or owned stock in F. Hoffman‐La Roche.

Supporting information

Data S1.

Data Availability Statement

For up‐to‐date details on Roche's Global Policy on the Sharing of Clinical Information and how to request access to related clinical study documents, see here: https://go.roche.com/data_sharing. For eligible studies, qualified researchers may request access to the clinical data through a data request platform. At the time of writing, this request platform is Vivli. https://vivli.org/ourmember/roche/.