Tackling dynamic prediction of death in patients with recurrent cardiovascular events

Abstract

In the field of cardiovascular disease, recurrent events such as stroke or myocardial infarction (MI) are often encountered, leading to an increase in the risk of death. Accurately evaluating the prognosis of patients and dynamically predicting the risk of death by considering the historical recurrent events can improve medical decisions and lead to better health care outcomes. Recently proposed joint modeling approaches within the Bayesian framework have inspired the development of a dynamic prediction tool, which can be applied for subject-level prediction of death with implementation in software packages. The prediction model incorporates subject heterogeneity with subject-level random effects that account for unobserved time-invariant factors and an extra copula function capturing the part caused by unmeasured time-dependent factors. Thereafter, given the prespecified landmark time $t^{\prime }$, the survival probability for a prediction horizon time of interest $t$ can be estimated for each individual. The prediction accuracy is assessed by time-dependent receiving operating characteristic curve and the area under the curve and the Brier score with calibration plots is compared to traditional joint frailty models. Finally, the tool is applied to patients with multiple attacks of stroke or MI in the Cardiovascular Health study and the Atherosclerosis Risk in Communities study for illustration.

1 |. INTRODUCTION

Recurrent events are common in medical practice and public health follow-up studies. In the field of cardiovascular disease (CVD), such events include heart attacks, stroke, and myocardial infarction, among others. Despite the known efficacy of existing therapies, such as statins, antiplatelet agents, renin-angiotensin-aldosterone system antagonists, and beta-blockers, for reducing CVD risk, cardiovascular disease remains a leading cause of death in the United States. The observation of these recurrences can indicate the status of disease progression or the overall health condition of the patients. Previous studies have shown that experiencing the first or subsequent episodes of disease events could be associated with a higher risk of mortality, leading to informative censoring.^1–3 Therefore, developing models for predicting disease progression may help identify patients with a high risk of mortality for more efficient and personalized medication. However, risk prediction in CVD is still under development, with limited work taking the history of CVD event occurrences into account.

When analyzing recurrent time-to-event data, frailty models are commonly used, where a random effect with a variety of distribution assumptions (eg, Gamma, log-normal) is considered to quantify the correlation between recurrences from the same subject. Copulas are also often used to describe the dependence between random variables, and this concept has been extended to modeling multivariate time-to-event processes.⁴ In the presence of a terminal event along with potential longitudinal measures, joint frailty models with multilayer random effects shared by different data components are used. The general idea is that data components are assumed to be independent of each other, conditioned on the shared random variables, and hence the correlation within the data structure is explained by the shared random variables. There is a substantial amount of work in this field. For example, Huang and Wang⁵ proposed a shared subject-level frailty in the hazard functions of recurrent gap times and the terminal event; Rondeau et al⁶ extended the shared frailty model by incorporating spline model for baseline intensity function; other extension can also be found in the literature.^7–9 Among them, Sinha et al⁷ and Ouyang et al⁹ applied Bayesian methods with an emphasis on modeling the risk of death and rejections for heart transplantation patients. Yu et al¹⁰ modeled the intensity of the recurrent events and the intensity of survival jointly while assuming independent censoring before death. Tallarita et al¹¹ and Paulon et al¹² proposed Bayesian non-parametric models for the gap times, with the latter one considering dependence with survival time. Most of this work assumes that the terminal event and recurrent event are independent, conditioned on a constant individual-level frailty. However, this assumption can be highly violated in reality.¹³ For instance, the dependency may depend on time-varying covariates (eg, environmental or socioeconomic factors) that are not fully captured by this subject-level frailty. Most recently, Emura et al^14,15 proposed a copula-based frailty model designed for meta-analysis in particular and that can be further extended to recurrent event models with an informative terminal event presented. However, their model is only valid when the terminal event process is a renewal process and the correlation remains constant over time. To address these gaps, we developed a flexible Bayesian framework that embedded the copula into joint frailty modeling of terminal and recurrent event processes.^16,17 In our model, a copula function is invoked to link the conditional processes of recurrent and terminal events to relax the assumptions of renewal process and conditional independence. Additionally, the richness in terms of choices of copula functions expands the flexibility and application of our model.

Establishing models of recurrent events and exploring their relationship with mortality is the first step in unraveling the complicated disease mechanisms. An immediate follow-up question will be how to assess the predictive accuracy of these models and identify an optimal one for clinical practice. An ideal prediction is not a one-time shot, but should be dynamically updated with the most up-to-date data and past history until a terminal event (ie, death). However, majority of the existing work use only a small portion of recurrent events information or disregard their historical records, thus dampening the predictive accuracy.^18–20 Since chronic diseases like CVD are dynamic, with disease severity and progression rates changing over time even for the same patient, personalized prediction of survival risk becomes necessary, especially under a dynamic scheme. In this work, we aim to conduct personalized prediction of survival risk by using our newly developed methods that can precisely capture the historical recurrent process compared to the other alternatives. We focus on two large observational CVD studies, the Cardiovascular Health Study (CHS), and the Atherosclerosis Risk in Communities study (ARIC), providing ample data resources for internal and external validation of survival prediction. Specifically, we predict all-cause mortality based on the observed history of recurrent CVD events of stroke and MI. The paper is organized as follows: Section 2 outlines the established models and procedures for dynamic prediction with predictive accuracy evaluation. We evaluate the numerical performance of our methods in Section 3, using extensive simulation studies. In Section 4, real data examples are provided for illustration of our approach. Finally, we discuss our results in Section 5.

2 |. METHOD

2.1 |. Notation

Let $i,j$ denote the index of the subject and the recurrent events, respectively. Let $D_i$ and $C_i$, respectively, denote the time to the terminal event and the censoring time for the ith subject, $i=1,\dots ,n$. Define ${\mathrm{\Delta }}_i=I\left(D_i<C_i\right)$ to be the failure indicator for the terminal event and $Y_i=\min \left\{D_i,C_i\right\}$ to be the observed follow-up time. Let $R_{ij}$ represent the gap time between the (j − 1)th and the jth recurrent events, and $T_{ij}={\sum }_{j^{\prime }=1}^j{R}_{i{j}^{\prime }}$ be the calendar time of the jth recurrent event with $j=1,\dots ,n_i$ and $T_{i0}$ defined as 0. Let $C_{ij}=\max \left\{Y_i-T_{i,j-1},0\right\}$ denote the censoring time for the jth gap time, $R_{ij}$, of recurrent events. Of note, all recurrent gap times are observed except that the last one is censored or stopped by the terminal event, that is, ${\delta }_{ij}=1$ for $j=1,\dots ,n_i$ and ${\delta }_{i,n_i+1}=0$. Note that $n_i\ge 0$ is assumed to be up to a sufficient large constant integer $J$, that is, a fixed maximum number of recurrent events that a subject can have, as described in our previous paper.^16,17 Also, $C_{ij}$ could depend on $R_{ij}$, indicating dependent censoring due to the existence of competing risk of the terminal event. Therefore, a joint modeling approach is considered and implemented here.

Next, we will briefly review our most recent developed joint hierarchical models that incorporate survival copulas under the joint frailty framework.

2.2 |. Joint hierarchical models

To date, there have been numerous studies in the literature that analyze recurrent events with or without the terminal event using joint frailty modeling.^10,21,22 However, most of these studies suffer from several limitations, including the strong conditional independence assumption, which may not hold in practical settings, as well as time-constant dependence between the processes of recurrent events and a terminal event. To address these concerns, we have recently developed novel joint hierarchical models known as joint hierarchical copula modeling (JHCM). This approach embeds survival copulas into frailty modeling and provides options for assuming constant or recurrence-dependent correlation between the two event processes.^16,17 In joint modeling, the hazard functions of recurrent gap time $R_{ij}$ and terminal event time $D_i$ are

$$\begin{array}{l}{h}_R\left(r\mid {\omega }_i\right)=h_{0R}(r)\exp \left\{{\mathit{\beta }}_R^T{\mathit{x}}_{i,R}+{\omega }_i\right\},\\ h_D\left(d\mid {\omega }_i\right)=h_{0D}(d)\exp \left\{{\mathit{\beta }}_D^T{\mathit{x}}_{i,D}+\psi {\omega }_i\right\},\end{array}$$ (1)

where ${\mathit{x}}_{i,D}$ and ${\mathit{x}}_{i,R}$ are time invariant or time-dependent covariates. ${\omega }_i\sim 𝒩\left(0,{\sigma }_{\omega }^2\right)$ is the subject-level frailty accounting for inter-recurrence correlation and also induces the dependency between recurrent events and death quantified by $\psi$. Under classic joint frailty models, the conditional survival distribution of $D_i$ and $R_{ij}$ given the frailty ${\omega }_i$ is $\mathrm{Pr}\left(D_i\ge d_i,R_{ij}\ge \right.\left.r_{ij}\mid {\omega }_i\right)=S_D\left(d_i\mid {\omega }_i\right)S_R\left(r_{ij}\mid {\omega }_i\right)$ for $i=1,\dots ,n$; $j=1,\dots ,n_i$. While for our JHCM, this conditional survival distribution is given by

$$\mathrm{Pr}\left(D_i\ge d_i,R_{ij}\ge r_{ij}|{\omega }_i\right)=C_{{\theta }_{ij}}\left(S_D\left(d_i|{\omega }_i\right),S_R\left(r_{ij}|{\omega }_i\right)\right),$$ (2)

where $C_{{\theta }_{ij}}(\cdot ,\cdot )$ is a time-dependent survival copula function with the parameter ${\theta }_{ij}$ quantifying the temporal heterogeneity on the relationship between the terminal event $D_i$ and the jth recurrent event time $R_{ij}$, while ${\omega }_i$ provides an overall measure of between-subjects heterogeneity induced by time-independent factors. In our first work by Li et al,¹⁶ a relatively simple assumption with a constant correlation was considered, that is, ${\theta }_{ij}=\theta$. Later on, in our second work by Li et al,¹⁷ this model was extended to allow the correlation between the terminal event and the recurrent event could vary across subject and event recurrence. To distinguish these two models, we name them as JHCM and time-varying JHCM (TVJHCM), respectively. Regarding to TVJHCM, it is recognized that the support constraint for survival copula coefficients may lead to modeling challenges, that is, ${\theta }_{ij}$ could be restricted to an interval, for example, for the Clayton Copula, ${\theta }_{ij}>0$, and for the Frank Copula, ${\theta }_{ij}\ne 0$. Initiated by ideas on hierarchical models,²³ we proposed to model ${\theta }_{ij}=g\left({\theta }_{\mu },{\gamma }_{ij}\right)$, with ${\theta }_{\mu }$ as an overall correlation between $D_i$ and $R_{ij}$ across recurrence, an unrestricted latent effect ${\gamma }_{ij}\in (-\infty ,\infty )$ and a pre-specified continuous function $g(\cdot )$. For instance, for Clayton and Frank Copula, we choose $g\left({\theta }_{\mu },{\gamma }_{ij}\right)={\theta }_{\mu }\exp \left({\gamma }_{ij}\right)$; for Gaussian Copula, $g\left({\theta }_{\mu },{\gamma }_{ij}\right)={\theta }_{\mu }\frac{\exp \left(2{\gamma }_{ij}\right)-1}{\exp \left(2{\gamma }_{ij}\right)+1}$. The other choices can be referred to Li et al. ¹⁶ The quantity of ${\theta }_{ij}$ and their dependence can be reflected through ${\gamma }_{ij}$ that are assumed to follow a Gaussian AR(1) process). Let ${\gamma }_i=\left\{{\gamma }_{i1},\dots ,{\gamma }_{i,n_i+1}\right\}$, which follows a multivariate normal distribution, denoted by $\mathbb{M}\mathbb{V}\mathbb{N}\left(0,{\text{Σ}}_i\right)$ with

$${\text{Σ}}_i=\frac{{\sigma }_{ϵ}^2}{1-{\rho }^2}\left(\begin{array}{cccc}1& \rho & \cdots & {\rho }^{n_i}\\ \rho & 1& \cdots & {\rho }^{n_i-1}\\ ⋮& ⋮& \ddots & ⋮\\ {\rho }^{n_i}& {\rho }^{n_i-1}& \cdots & 1\end{array}\right).$$

Based on our assumptions on terminal and recurrent events and potential data patterns, we have joint survival probabilities conditionally on ${\omega }_i$ below

$$\begin{array}{l}\mathrm{Pr}\left(D_i\ge y_i,R_{ij}\ge r_{ij}\mid {\omega }_i\right)=C\left(S_D\left(y_i\mid {\omega }_i\right),S_R\left(r_{ij}\mid {\omega }_i\right)\right)\\ \mathrm{Pr}\left(D_i\ge y_i,R_{ij}=r_{ij}\mid {\omega }_i\right)=C_{[01]}\left(S_D\left(y_i\mid {\omega }_i\right),S_R\left(r_{ij}\mid {\omega }_i\right)\right)f_R\left(r_{ij}\mid {\omega }_i\right)\\ \mathrm{Pr}\left(D_i=y_i,R_{ij}\ge r_{ij}\mid {\omega }_i\right)=C_{[10]}\left(S_D\left(y_i\mid {\omega }_i\right),S_R\left(r_{ij}\mid {\omega }_i\right)\right)f_D\left(y_i\mid {\omega }_i\right)\\ \mathrm{Pr}\left(D_i=y_i,R_{ij}=r_{ij}\mid {\omega }_i\right)=C_{[11]}\left(S_D\left(y_i\mid {\omega }_i\right),S_R\left(r_{ij}\mid {\omega }_i\right)\right)f_D\left(y_i\mid {\omega }_i\right)f_R\left(r_{ij}\mid {\omega }_i\right),\end{array}$$

where $C(\cdot )$ is a prespecified survival copula function accounting for the correlation between two event processes, $f(\cdot )$ is denoted as a density function, $C_{[01]}(\mu ,v)=\partial C(\mu ,v)/\partial v$, $C_{[10]}(\mu ,v)=\partial C(\mu ,v)/\partial \mu$, $C_{[11]}(\mu ,v)={\partial }^{2}C(\mu ,v)/\partial \mu \partial v$. For all $n$ subjects, we denote the observed data as ${𝒟}_n=\left\{y_i,r_{ij},{\text{Δ}}_i,{\delta }_{ij},{\mathit{x}}_{i,R},{\mathit{x}}_{i,D},i=1,\dots ,n;j=1,\dots ,n_i\right\}$. Thereafter, the conditional likelihood given the frailty $\mathit{\omega }$ can be expressed by

$$\begin{array}{l}L\left({𝒟}_n\mid \mathit{\omega }\right)=\prod _{i=1}^n{h}_D^{{\mathrm{\Delta }}_i}\left(y_i\mid {\omega }_i\right)S_D\left(y_i\mid {\omega }_i\right)\\ \times \prod _{j=1}^{n_i+1}{\left[\frac{C_{[01]}\left(S_D\left(y_i\mid {\omega }_i\right),S_R\left(r_{ij}\mid {\omega }_i\right)\right)f_R\left(r_{ij}\mid {\omega }_i\right)}{S_D\left(y_i\mid {\omega }_i\right)}\right]}^{\left(1-{\mathrm{\Delta }}_i\right){\delta }_{ij}}\\ \times {\left[C_{[11]}\left(S_D\left(y_i\mid {\omega }_i\right),S_R\left(r_{ij}\mid {\omega }_i\right)\right)f_R\left(r_{ij}\mid {\omega }_i\right)\right]}^{{\mathrm{\Delta }}_i{\delta }_{ij}}\\ \times {\left[C_{[10]}\left(S_D\left(y_i\mid {\omega }_i\right),S_R\left(r_{ij}\mid {\omega }_i\right)\right)f_R\left(r_{ij}\mid {\omega }_i\right)\right]}^{{\mathrm{\Delta }}_i\left(1-{\delta }_{ij}\right)}\\ \times {\left[\frac{C_{[00]}\left(S_D\left(y_i\mid {\omega }_i\right),S_R\left(r_{ij}\mid {\omega }_i\right)\right)}{S_D\left(y_i\mid {\omega }_i\right)}\right]}^{\left(1-{\mathrm{\Delta }}_i\right)\left(1-{\delta }_{ij}\right)}.\end{array}$$

Note that depending on the parameter set-up for the copula function linking the two conditional time-to-event processes in JHCM and TVJHCM, the expression of the conditional likelihood can be slightly different.

For parameter estimation and inference, we adopt the Bayesian techniques (ie, the Metropolis-Hastings within the Gibbs sampler algorithm).²⁴ Non-informative priors are considered in Bayesian inference for all the parameters in the model. Regarding to baseline hazards, we have multiple choices, including constant hazards, specific parametric function (eg, Weibull) or commonly used piece-wise functions. Noting that for piece-wise constant hazards, it is crucial to determine the number and the lengths of intervals of the base line risk function. Here, because of simplicity and data generation in numerical studies, we currently consider constant hazards for method illustration, ie, $h_{0R}(r)=\exp \left({\beta }_{0R}\right)$, $h_{0D}(d)=\exp \left({\beta }_{0D}\right)$. Thus, define $\mathbf{\Theta }=\left\{{\beta }_{0R},{\beta }_{0D},{\mathit{\beta }}_D,{\mathit{\beta }}_R,\mathit{\xi }\right\}$ as the parameter space, where $\mathit{\xi }=\left\{\theta ,{\sigma }_{\omega }^2\right\},\mathit{\xi }=\left\{{\theta }_{\mu },{\sigma }_{\omega }^2,{\sigma }_{ϵ}^2,\rho \right\}$ in JHCM and TVJHCM, respectively. Let Gamma(⋅) denote a gamma distribution, $IG(\cdot )$ an inverse gamma distribution and $U(\cdot )$ a uniform distribution. Thus, we have

$$\begin{array}{c}\mathrm{Pr}\left({\beta }_{0D}\right)\propto 1,\mathrm{Pr}\left({\beta }_{0D}\right)\propto 1,\mathrm{Pr}\left({\mathit{\beta }}_R\right)\propto \mathbf{1},\mathrm{Pr}\left({\mathit{\beta }}_R\right)\propto \mathbf{1},\\ {\theta }_{\mu }\sim \text{Gamma}(\alpha ,\beta ),{\sigma }_{\omega }^2\sim IG(\alpha ,\beta ),{\sigma }_{ϵ}^2\sim IG(\alpha ,\beta ),\rho \sim U(-1,1),\end{array}$$

where $\alpha =0.001$ and $\beta =1000$ to ensure that the prior is non-informative. Take TVJHCM as an example to describe some technical details, we treat $\mathit{\omega }$ and $\mathit{\gamma }$ as unobserved quantities and adopt the Bayesian data augmentation algorithm (including I-step for missing data imputation and P-step for drawing posterior samples) to handle this missing data problem.²⁴ Here, we take $\mathit{\omega }$ for an illustration of the algorithm. Given ${\omega }_{-i}$ is defined as the random terms $\mathit{\omega }$ except ${\omega }_i$ in the I-step, we sample ${\omega }_i$ from $\mathrm{Pr}\left({\omega }_i\mid {\mathit{\omega }}_{-i},\mathit{\gamma },\mathbf{\Theta },{𝒟}_n\right)\propto ℒ\left(\mathbf{\Theta }\mid \mathit{\omega },\mathit{\gamma },{𝒟}_n\right)\mathrm{Pr}\left({\omega }_i\mid {\mathit{\omega }}_{-i},{\sigma }_{\omega }^2\right)$. In the P-step, we have ${\sigma }_{\omega }^2\mid \mathit{\omega },{𝒟}_n\sim IG(\alpha +n/2,{\{{\beta }^{-1}+0.5{\sum }_{j=1}^n{\omega }_j^2\}}^{-1})$. The algorithm’s step size is chosen to be around 0.01 to achieve an acceptance rate of approximately 40%. Further information about this procedure, as well as the JHCM approach, can be found in our published work by Li et al.^16,17 Previous research on empirical evaluation has recognized the potential of our proposed models for reducing bias and improving efficiency compared to existing alternative methods.^16,17 However, their predictive performance has not yet been exclusively assessed, which could be crucial for making treatment decisions in clinical practice.

2.3 |. Dynamic prediction of mortality

Predicting mortality based on the observed history of recurrent events is essential for personalized medicine with improved health care outcomes. However, this field of study is still in its early stages. In this study, we perform dynamic prediction of mortality using JHCM, TVJHCM, and traditional JFM models for comprehensive evaluation and comparison. Our goal is to predict the probability of death within a specific time window between a prespecified time point $t^{\prime }$ and a horizon time point $t$, given patient history data up to time $t^{\prime }\left(t>t^{\prime }\right)$. Next, we will introduce two types of validation procedures.

For internal validation, we use the same data to build the prediction model and evaluate its accuracy. To prevent overfitting, we use the random sub-resampling validation method.²⁵ Basically, the data is randomly split into two equal-sized sets of subjects. The model is built on one set, and the estimated parameters are used to predict the probabilities of death for patients in the other set. This procedure is repeated B times (ie, B = 40). At the end of this process, each patient who is alive and uncensored at time $t^{\prime }$ will have a predicted probability of death. We also consider the K-fold cross-validation approach, but we find that it results in less satisfactory performance in terms of relatively larger prediction errors. This could be due to the relatively small sample sizes or imbalanced data resulting from highly varied numbers of recurrent events across subjects. Therefore, here we mainly focus on the former approach.

Moreover, we conduct external validation by using independent training and testing datasets denoted by $𝒟\mathit{train}$ and $𝒟\mathit{test}$ respectively. The prediction model will be built using the training data $𝒟\mathit{train}$, and its accuracy is evaluated using the testing data $𝒟\mathit{test}$. Similarly, we estimate the probability of death for each “new” subject in the testing data who is alive and uncensored at time $t^{\prime }$.

Now, we briefly describe how to obtain the predicted probability of death for each subject. Suppose $\mathbf{\Theta }$ as the set including all the parameters in the predictive candidate model with the estimates denoted by $\widehat{\mathbf{\Theta }}$. Let $N_{t^{\prime }}$ denote the number of subjects that are alive and uncensored at time $t^{\prime }$. For subject $i\left(i=1,\dots ,N_{t^{\prime }}\right)$, the accumulative observed recurrent events history up to $t^{\prime }$ is denoted by ${\mathscr{H}}_i\left(t^{\prime }\right)=\{r_{i,1},\dots ,r_{i,j},s.t.{\sum }_{j^{\prime }=1}^j{r}_{i,j^{\prime }}<t^{\prime }\text{and}{\sum }_{j^{\prime }=1}^{j+1}{r}_{i,j^{\prime }}>t^{\prime }\}$. We aim to estimate the conditional survival probability, $\pi \left(t\mid t^{\prime },{\mathscr{H}}^{*}\left(t^{\prime }\right),{\omega }_i\right)=\mathrm{Pr}\left(D_i>t\mid D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,\mathbf{\Theta },{\omega }_i\right)=\frac{\mathrm{Pr}\left(D_i>t\mid {\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,\mathbf{\Theta },{\omega }_i\right)}{\mathrm{Pr}\left(D_i>t^{\prime }\mid {\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,\mathbf{\Theta },{\omega }_i\right)}$ under the Bayesian framework. More details about the estimation algorithm (Algorithm 1) are provided in Section 2.4.4. Next, we discuss the metrics for predictive accuracy.

2.4 |. Evaluation of prediction accuracy

Several metrics exist in the literature for evaluating the predictive performance of time-to-event models. For our clinical purposes and simplicity, we emphasize calibration, which quantifies how well the model predicts the outcome, based on the Brier score (BS).^26,27 We also evaluate discrimination ability, which assesses how well the model distinguishes between subjects with and without the event.²⁸ The predictive accuracy metric of the BS is defined by

$$E\left\{{\left[D\left(t\mid t^{\prime },\mathscr{H}\left(t^{\prime }\right)\right)-\widehat{\pi }\left(t\mid t^{\prime },\mathscr{H}\left(t^{\prime }\right);\widehat{\mathbf{\Theta }}\right)\right]}^2\right\},$$ (3)

where $D\left(t\mid t^{\prime },\mathscr{H}\left(t^{\prime }\right)\right)$ is the observed survival status coded by 1 if the subject is still alive by $t$ and otherwise 0, $\widehat{\pi }(t\mid t^{\prime },\mathscr{H}\left(t^{\prime }\right);\widehat{\mathbf{\Theta }})$ is the estimated survival probability by $t$, and $\mathscr{H}\left(t^{\prime }\right)$ denotes the history data up to $t^{\prime }$. The inverse of probability of censoring weighting (IPCW)-based estimate of the BS is calculated for empirical prediction assessment.^16,26,27 Note that the censoring weight denoted by $G_i\left(t^{\prime },t\right)$ can be estimated by ${\widehat{G}}_i\left(t^{\prime },t\right)=I\left(d_i>t\right)/\{{\widehat{S}}_0(t)/{\widehat{S}}_0\left(t^{\prime }\right)\}+I\left(t>d_i>t^{\prime }\right){\delta }_i/\{{\widehat{S}}_0\left(d_i\right)/{\widehat{S}}_0\left(t^{\prime }\right)\}$, with $I(\cdot )$ as an indicator function and ${\widehat{S}}_0$ estimated based on the Kaplan-Meier curve.

In addition, we also consider the time-dependent receiving operating characteristic (ROC) curve and its area under the curve (AUC) for evaluation.²⁹ The time-dependent ROC curve is constructed using the cumulative sensitivity denoted by $\mathrm{Sens}(c,t)$ and dynamic specificity denoted by $\mathrm{Spec}(c,t)$, where $c$ is a prespecified threshold. The ROC-AUC at time $t$ is computed based on the estimated risk ${\pi }_i^{*}=1-{\widehat{\pi }}_i(\widehat{\mathbf{\Theta }})$, as follows:

$$\mathrm{AUC}(t)=\mathrm{Pr}({\pi }_i^{*}>{\pi }_j^{*}\mid D_i\le t,D_j>t)=\int \mathrm{Sens}(c,t)d[1-\mathrm{Spec}(c,t)],i\ne j.$$ (4)

This metric can be interpreted as the probability that a randomly selected case whose death is before time $t$ has a higher predicted risk than a randomly selected control whose death is after time $t$.

Moreover, as per the reviewers’ suggestion, we perform more calibration assessment by generating calibration plots, which are generated by the following steps: (1) grouping the predicted risks of death into deciles based on their distribution, (2) plotting the observed proportions of terminal events against the mean predicted probabilities for each decile. These plots also feature histograms of the predicted probabilities to illustrate their distribution.

Next, we will provide technical details regarding the estimation of predictive probabilities for the candidate models, including JHCM, TVJHCM, and JFM.

2.4.1 |. Prediction based on JHCM

For JHCM, we assume a constant and singular parameter for the copula function, and the parameter space is $\mathbf{\Theta }=\left\{{\beta }_{0R},{\beta }_{0D},{\mathit{\beta }}_D,{\mathit{\beta }}_R,{\sigma }_{\omega }^2,\theta \right\}$. As described above, the parameter estimates can be obtained based on the Bayesian technique. Thus, we can get ${\widehat{\pi }}_i\left(t\mid t^{\prime }\right)=\mathrm{Pr}(D_i>t\mid D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right),D_n,{\mathit{x}}_i,\widehat{\mathbf{\Theta }},{\widehat{\omega }}_i)$ for “new” subject $i$, where ${\mathit{x}}_i$ includes subject-specific covariates of interest, $\widehat{\mathbf{\Theta }}$ and ${\widehat{\omega }}_i$ are the plugged-in posterior means. In particular, in order to get ${\widehat{\omega }}_i$ from “new” subject $i$, we need to adopt the Metropolis–Hastings (MH) within Gibbs Sampler algorithm for sampling data from the conditional posterior distribution, that is

$$\mathrm{Pr}\left({\omega }_i\mid D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,{\mathit{x}}_i,\mathbf{\Theta }\right)\propto \mathrm{Pr}\left(D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right)\mid {𝒟}_n,{\mathit{x}}_i,\mathbf{\Theta },{\omega }_i\right)Pr\left({\omega }_i\mid \mathbf{\Theta },{𝒟}_n\right)Pr\left(\mathbf{\Theta }\mid {𝒟}_n\right).$$ (5)

2.4.2 |. Prediction based on TVJHCM

The prediction under TVJHCM can be done along with the same procedure as JHCM but with the parameter space $\mathbf{\Theta }=\left\{{\beta }_{0R},{\beta }_{0D},{\mathit{\beta }}_D,{\mathit{\beta }}_R,{\sigma }_{ϵ}^2,{\sigma }_{\omega }^2,{\theta }_{\mu },\rho \right\}$. For “new” subject $i$, we have subject-specific latent effects ${\gamma }_i={\left({\gamma }_{i1},{\gamma }_{i2},\dots ,{\gamma }_{ij}\right)}^T$ besides the frailty ${\omega }_i$. The survival probability, ${\pi }_i\left(t\mid t^{\prime }\right)$, is defined by $\mathrm{Pr}\left(D_i>t\mid D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,{\mathit{x}}_i,\mathbf{\Theta },{\omega }_i,{\mathit{\gamma }}_i\right)$, which can be estimated after plugging in the posterior mean estimates of $\widehat{\mathit{\Theta }}$, ${\widehat{\omega }}_i$ and ${\widehat{\mathit{\gamma }}}_i$. In particular, in order to get ${\widehat{\omega }}_i$ from “new” subject $i$, the conditional posterior distributions are expressed as

$$\begin{array}{l}Pr\left({\omega }_i\mid D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,{\mathit{x}}_i,{\mathit{\gamma }}_i,\mathbf{\Theta }\right)\\ \propto Pr\left(D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right)\mid {𝒟}_n,{\mathit{x}}_i,\mathbf{\Theta },{\omega }_i,{\mathit{\gamma }}_i\right)Pr\left({\omega }_i\mid \mathbf{\Theta },{𝒟}_n\right)Pr\left({\mathit{\gamma }}_i\mid \mathbf{\Theta },{𝒟}_n\right)Pr\left(\mathbf{\Theta }\mid {𝒟}_n\right).\end{array}$$ (6)

In addition, for ${\widehat{\mathit{\gamma }}}_i$, we need to update each element by utilizing the conditional posterior distribution for ${\mathit{\gamma }}_{ij}$ given ${\gamma }_{i,-j}=\left\{{\gamma }_{i1},\dots ,{\gamma }_{i,j-1},{\gamma }_{i,j+1},\dots ,{\gamma }_{i,n_i+1}\right\}$, which is shown as

$$\begin{array}{l}Pr\left({\gamma }_{ij}|{𝒟}_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,{\mathit{x}}_i,{\mathit{\gamma }}_{i,-j},{𝒟}_i,\mathbf{\Theta }\right)\\ \propto Pr\left({𝒟}_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right)|{𝒟}_n,{\mathit{x}}_i,{\omega }_i,{\mathit{\gamma }}_i,\mathbf{\Theta }\right)Pr\left({\mathit{\gamma }}_{ij}|{\mathit{\gamma }}_{i,-j},\mathbf{\Theta },{𝒟}_n\right)Pr\left({\omega }_i|\mathbf{\Theta },{𝒟}_n\right)Pr\left(\mathbf{\Theta }|{𝒟}_n\right).\end{array}$$ (7)

2.4.3 |. Prediction based on JFM

JFM is a widely used joint modeling approach, under which the processes of recurrent and terminal events are conditionally independent given the subject-level frailty ${\omega }_i$. For the sake of comparisons with our proposed models, we checked the accuracy of dynamic prediction under this framework. Under JFM, the parameter space $\mathbf{\Theta }=\left\{h_{0R},h_{0D},{\mathit{\beta }}_D,{\mathit{\beta }}_R,{\sigma }_{\omega }^2,\theta \right\}$, where $\theta$ is the coefficient for the frailty term ${\omega }_i\sim N\left(0,{\sigma }_{\omega }^2\right)$ that controls the association strength between recurrent and terminal event processes, and the other parameters are defined the same as above. The prediction can be performed using the R package of frailtypack,^30,31 where the survival probability at $t$ can be derived as

$$\begin{array}{l}Pr\left(t,t^{\prime };\mathbf{\Theta }\right)=Pr\left(D_i>t\mid D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,{\mathit{x}}_i,\mathbf{\Theta }\right)\\ ={\int }_0^{\infty }P\left(D_i>t\mid D_i>t^{\prime },{\mathscr{H}}_i\left(t^{\prime }\right),{𝒟}_n,{\mathit{x}}_i,{\omega }_i,\mathbf{\Theta }\right)g\left({\omega }_i\mid \mathbf{\Theta }\right)d{\omega }_i.\end{array}$$ (8)

2.4.4 |. The generic algorithm for prediction of mortality

Algorithm 1.

DYNAMIC PREDICTION OF MORTALITY GIVEN THE HISTORY OF RECURRENT EVENTS

Step 1: Obtain the Bayesian estimates of $\mathbf{\Theta }$ using the training data ${𝒟}_{\text{train}}$ and establish the prediction model

Step 2: Predict survival probabilities at a pre-specified sequence of time points $\left\{t_1^{\prime },\dots ,t_L^{\prime }\right\}$ using the testing data ${𝒟}_{\text{test}}$ given the historical data up to $t^{\prime }$. For instance in our simulation, $\left\{t_1^{\prime },\dots ,t_L^{\prime }\right\}$ is a sequence from $t^{\prime }$ to 0.1 by 0.01, which is subjectively determined depending on the clinical need and sample size. The procedure is listed below: for each $t\in \left\{t_1^{\prime },\dots ,t_L^{\prime }\right\}$;

for each “new” subject $i$ within the risk set $R_t$ at time $t$;

Set an initial point ${\omega }_i^{(0)}$ and ${\mathit{\gamma }}_i^{(0)}$

for iteration $k=1,2,\dots ,K$ (a relatively large number, eg, 1000)

(a) Generate samples of ${\omega }_i$ from posterior distributions (Equation 5 or 6)

(b) Generate samples of ${\gamma }_{ij}$ from posterior distributions (Equation 7)

until iteration $K$ is done based on the MH with Gibbs sampling algorithm. Thereafter, calculate the survival probability ${\pi }_i\left(t\mid t^{\prime }\right)$ defined in Section 2.4.1 and 2.4.2 by plugging in the posterior sample means of ${\widehat{\omega }}_i$ and ${\widehat{\mathit{\gamma }}}_i$

end for

end for

end for

if the event occurs with the observed time

Estimate the BS (Equation 3) and the ROC-AUC (Equation 4);

end if

3 |. SIMULATION STUDIES

3.1 |. Simulation setup

To evaluate the prediction accuracy and robustness of JHCM and TVJHCM in comparison to traditional JFM, we perform extensive simulation studies. We consider various scenarios with a range of parameter setups and assessed the models’ performance under each scenario. For each Monte Carlo replication, we randomly split the data with the total sample size of 200 and 800 into two sets: a training set $𝒟\mathit{train}$ and a testing set $𝒟\mathit{test}$, both with equal sample sizes of $n=100$ and $n=400$. The total sample size was 200 or 800, depending on the scenario.

First, we consider the scenario with time-varying dependency between the event processes of recurrences and death.

For the ith subject $\text{(}i=1,\dots ,n\text{)}$, the hazard functions of recurrent events and death are

$$h_R\left(r_{ij}\right)={\beta }_{0R}\exp \left(x_{i1}{\beta }_{R,1}+x_{i2}{\beta }_{R,2}+{\omega }_i\right);$$ (9)

$$h_D\left(d_i\right)={\beta }_{0D}\exp \left(x_{i1}{\beta }_{D,1}+x_{i2}{\beta }_{D,2}+{\omega }_i\right),$$ (10)

where $x_{i1}$ is generated from a standard normal distribution and $x_{i2}$ is from a Bernoulli distribution with the probability of 0.5. Also, constant baseline hazards ${\left({\beta }_{0D},{\beta }_{0R}\right)}^T={(0.5,1)}^T$ and regression parameters ${\left({\beta }_{D,1},{\beta }_{D,2},{\beta }_{R,1},{\beta }_{R,2}\right)}^T={(1,1,2,2)}^T$. The frailty term ${\omega }_i$ follows a normal distribution with mean 0 and variance ${\sigma }_{\omega }^2=0.5$. The detailed data generation procedures are shown below:

• Step 1: Generate a random variable $D_i$ by $-\log \left(U_i\right)/h_D$, where $h_D$ is shown in (10), and $U_i=S_D\left(d_i\mid {\omega }_i\right)$ is generated from $U(0,1)$.

• Step 2: Generate an independent censoring time $C_i$ from a uniform distribution, $Unif(0,c)$, where $c=1,0.6,0.4$ to ensure that the death rate is around 60%, 50%, or 40%, respectively;

• Step 3: Generate the observed follow-up time $Y_i=\min \left(D_i,C_i\right)$ where the death indicator $D_i$ and censoring indicator are generated in the previous two steps;

• Step 4: Given $V_{ij}=S_R\left(r_{ij}\mid {\omega }_i\right)$, the joint distribution of $U_i$ and $V_{ij}$ can be modeled using the prespecified Clayton copula function with the parameter ${\theta }_{ij}$. Noting that when $D_i\le C_i$, the conditional distribution of $V_{ij}$ is

  $$F_V\left(v\mid U_i=u_i\right)=u_i^{-\left({\theta }_{ij}+1\right)}{(u_i^{-{\theta }_{ij}}+v^{-{\theta }_{ij}}-1)}^{-\left({\theta }_{ij}+1\right)/{\theta }_{ij}}.$$ (11)

  When $D_i\le C_i$, the conditional distribution of $V_{ij}$ follows up a uniform distribution $Unif(0,1)$. Thus, we define ${\tilde{W}}_{ij}=F_V\left(v\mid U_i=u_i\right)$, which can be generated from $Unif(0,1)$. Thereafter, we generate $V_{ij}$ using the following equation:

  $$V_{ij}={\left(({\tilde{W}}_{ij}^{-\frac{{\theta }_{ij}}{{\theta }_{ij}+1}}-1)u_i^{-{\theta }_{ij}}+1\right)}^{-\frac{1}{{\theta }_{ij}}}.$$ (12)

  When $D_i>C_i$, we have $F_V\left\{v\mid U_i<\exp \left(-h_D{C}_i\right)\right\}$, then $V_{ij}$ can be generated based upon the Monte Carlo method.

• Step 5: Generate the gap time $r_{ij}$ by using $-log\left(V_{ij}\right)/h_R$, where $h_R$ is shown in (9). Repeat this procedure until $T_{ij}$ is greater than the observed follow-up time $Y_i$.

Note that in TVJHCM, the parameters for the Clayton copula $\left({\theta }_{ij}\right)$ are obtained using ${\theta }_{\mu }$ and the pairwise correlation $\rho$. We set ${\theta }_{\mu }$ and $\theta$ to 1, 1.5, or 2, J to 1000, and $\rho$ to 0.1, 0.3, or 0.5. We also consider three different censoring rates of 0.4, 0.5, or 0.6, resulting in a total of 81 parameter combinations for this scenario. Additionally, we consider a scenario with time-varying dependency between two event processes. Under this setting, we do not have $\rho$ in the parameter space, and ${\theta }_{ij}$ is set to $\theta =1$,1.5, or 2, with the other setups being the same as in the previous scenario.

For each Monte Carlo dataset, we fit JHCM, TVJHCM, and JFM models using the training data. Regarding parameter estimation, non-informative priors are considered, as described in Section 3. ${\sigma }_{\omega }^{-2}$ is assumed to follow a prior $\text{Gamma}(\alpha ,1/\alpha )$ with $\alpha =0.001$, so that the prior is flat enough. We also thin the posterior samples every 10 iterations to lower the dependence of the MCMC samples.

Based on the estimated candidate models from the training data, we estimate $\pi \left(t|t^{\prime }\right)$ for each subject in the testing data, where $t^{\prime }$ is set to 0.03, 0.06, or 0.09 (corresponding to the proportions of subjects at risk as 80%, 70%, and 60%, respectively), and $t$ takes values between $t^{\prime }$ and 0.1 with increments of 0.01. To summarize the results, we calculate the mean estimates and standard deviations (SD) of both BS and ROC-AUC metrics across 100 Monte Carlo datasets. Furthermore, we evaluate the robustness of our joint modeling by considering the mis-specification of copula functions. Specifically, we generate data from a Frank copula but evaluate model fitting and prediction using a mis-specified copula (eg, Clayton).

3.2 |. Simulation results

Tables 1 and 2 present the results of the BS for the scenarios with time-varying dependence between two event processes. The results show that TVJHCM outperforms JHCM and JFM (the worst) in terms of the smallest BS. Additionally, the BS tends to increase as the prediction time $t$ moves farther away from $t^{\prime }$, which agrees with our expectation. For instance, when $t^{\prime }=0.03$ (refer to Table 1), the BS is 0.02 and 0.07 for TVJHCM and JHCM, respectively, and also when $t$ increases to 0.1, the BS increases to 0.18 under TVJHCM, and to 0.24 under JHCM. Furthermore, we observe a lower BS for larger $t^{\prime }$ given a fixed sample size and prediction time $t$ for both TVJHCM and JHCM, which is likely due to the collection of more historical data for prediction. Additional findings include: (1) a larger sample size improves prediction accuracy, particularly for the case with a larger landmark time $t^{\prime }$. Additional simulation studies with an increased sample size of 800 are performed for verification (refer to the Data S1); (2) all models perform better as the censoring rate is lower, and more evidence of superiority of TVJHCM over JHCM and JFM can be shown; (3) the performance of JHCM deteriorates when time-varying dependency fluctuates widely between two event processes, especially for larger sample sizes. In such cases, TVJHCM’s advantages are evident in terms of the smallest BS.

TABLE 1.

Comparison of Brier score (BS) based on the time-varying JHCM (TVJHCM), joint hierarchical copula modeling (JHCM), and JFM when the parameter θ_μ = 1, ρ = 0.1.

		Censoring rate = 0.6			Censoring rate = 0.5			Censoring rate = 0.4
n	t	TVJHCM (SD)	JHCM (SD)	JFM (SD)	TVJHCM (SD)	JHCM (SD)	JFM (SD)	TVJHCM (SD)	JHCM (SD)	JFM (SD)
100	0.03	4.00 (2.35)	7.91 (2.36)	17.46 (2.39)	3.59 (2.24)	7.59 (2.30)	17.57 (2.34)	2.17 (2.21)	6.60 (2.25)	13.23 (2.33)
	0.04	7.73 (2.36)	11.61 (2.37)	19.85 (2.41)	7.19 (2.34)	11.01 (2.34)	19.80 (2.40)	5.84 (2.33)	9.94 (2.33)	15.21 (2.38)
	0.05	10.44 (2.37)	15.01 (2.39)	22.09 (2.44)	9.92 (2.35)	14.36 (2.35)	21.67 (2.43)	8.67 (2.34)	13.18 (2.34)	16.76 (2.42)
	0.06	12.63 (2.38)	18.14 (2.40)	23.80 (2.45)	12.23 (2.36)	17.33 (2.36)	23.13 (2.44)	11.24 (2.35)	15.94 (2.35)	18.16 (2.43)
	0.07	14.59 (2.39)	20.87 (2.41)	25.45 (2.46)	14.06 (2.37)	19.99 (2.37)	24.46 (2.45)	13.19 (2.36)	18.39 (2.36)	19.27 (2.44)
	0.08	16.00 (2.40)	23.08 (2.43)	26.58 (2.49)	15.62 (2.38)	22.23 (2.39)	25.53 (2.46)	15.00 (2.37)	20.47 (2.39)	20.35 (2.46)
	0.09	17.17 (2.43)	25.13 (2.47)	27.70 (2.54)	16.88 (2.40)	24.24 (2.45)	26.54 (2.51)	16.43 (2.38)	22.49 (2.42)	21.52 (2.50)
	0.10	18.04 (2.45)	26.93 (2.49)	28.64 (2.57)	17.80 (2.42)	26.15 (2.47)	27.51 (2.55)	17.90 (2.40)	24.20 (2.45)	22.53 (2.53)
400	0.03	4.03 (1.12)	7.99 (1.23)	16.92 (1.34)	3.48 (1.10)	7.52 (1.21)	15.83 (1.34)	2.09 (1.09)	6.34 (1.20)	11.26 (1.33)
	0.04	7.82 (1.12)	11.67 (1.23)	19.31 (1.34)	7.12 (1.10)	11.06 (1.21)	18.03 (1.34)	5.86 (1.09)	9.47 (1.20)	13.06 (1.33)
	0.05	10.83 (1.12)	15.03 (1.23)	21.22 (1.34)	10.07 (1.10)	14.25 (1.21)	19.75 (1.34)	9.10 (1.09)	12.33 (1.20)	14.58 (1.33)
	0.06	13.05 (1.14)	18.03 (1.26)	22.75 (1.38)	12.40 (1.13)	17.18 (1.26)	21.25 (1.39)	11.86 (1.10)	14.98 (1.24)	15.93 (1.36)
	0.07	14.75 (1.14)	20.86 (1.26)	24.15 (1.38)	14.08 (1.13)	19.85 (1.26)	22.52 (1.39)	14.03 (1.10)	17.55 (1.24)	17.06 (1.36)
	0.08	16.17 (1.16)	23.24 (1.26)	25.36 (1.41)	15.50 (1.14)	22.21 (1.26)	23.67 (1.40)	15.95 (1.13)	19.83 (1.25)	18.19 (1.40)
	0.09	17.23 (1.16)	25.40 (1.26)	26.37 (1.41)	16.76 (1.14)	24.29 (1.26)	24.65 (1.40)	17.73 (1.13)	22.01 (1.25)	19.35 (1.40)
	0.10	18.22 (1.16)	27.26 (1.26)	27.28 (1.41)	17.84 (1.14)	26.24 (1.26)	25.61 (1.40)	19.45 (1.13)	23.83 (1.25)	20.51 (1.40)
100	0.06	8.57 (2.31)	15.13 (2.33)	25.25 (2.39)	7.69 (2.29)	14.37 (2.31)	24.49 (2.37)	4.68 (2.26)	12.84 (2.29)	19.54 (2.35)
	0.07	11.53 (2.33)	17.93 (2.35)	27.07 (2.43)	10.45 (2.31)	17.06 (2.33)	26.04 (2.40)	7.40 (2.30)	15.26 (2.31)	20.95 (2.38)
	0.08	13.71 (2.35)	20.24 (2.38)	28.31 (2.46)	12.78 (2.33)	19.43 (2.35)	27.28 (2.43)	9.88 (2.31)	17.34 (2.34)	22.27 (2.42)
	0.09	15.55 (2.36)	22.57 (2.41)	29.52 (2.49)	14.71 (2.34)	21.77 (2.38)	28.42 (2.47)	11.84 (2.32)	19.50 (2.35)	23.63 (2.46)
	0.10	16.97 (2.38)	24.66 (2.43)	30.51 (2.52)	16.17 (2.36)	23.86 (2.41)	29.50 (2.50)	13.83 (2.34)	21.34 (2.40)	24.83 (2.50)
400	0.06	8.83 (1.13)	14.90 (1.25)	24.06 (1.37)	7.56 (1.13)	14.14 (1.25)	22.73 (1.37)	4.22 (1.09)	11.76 (1.22)	17.51 (1.35)
	0.07	11.51 (1.13)	17.73 (1.25)	25.63 (1.37)	10.12 (1.13)	16.81 (1.25)	24.23 (1.37)	7.11 (1.09)	14.14 (1.22)	19.15 (1.35)
	0.08	13.75 (1.14)	20.27 (1.26)	26.93 (1.38)	12.26 (1.14)	19.21 (1.26)	25.55 (1.38)	9.66 (1.10)	16.33 (1.22)	20.70 (1.35)
	0.09	15.49 (1.14)	22.64 (1.26)	28.01 (1.38)	14.16 (1.14)	21.45 (1.26)	26.66 (1.38)	12.02 (1.10)	18.48 (1.22)	22.23 (1.35)
	0.10	17.08 (1.15)	24.74 (1.26)	28.99 (1.40)	15.81 (1.13)	23.61 (1.25)	27.70 (1.39)	14.27 (1.09)	20.38 (1.20)	23.65 (1.38)
100	0.09	13.31 (2.21)	22.21 (2.23)	32.31 (2.33)	12.90 (1.99)	20.51 (2.20)	30.37 (2.32)	11.69 (1.98)	19.56 (2.19)	29.31 (2.30)
	0.10	15.51 (2.22)	23.41 (2.24)	33.41 (2.34)	14.97 (2.20)	22.72 (2.24)	31.34 (2.34)	13.74 (2.20)	21.72 (2.23)	30.39 (2.34)
400	0.09	13.03 (1.10)	19.98 (1.21)	28.79 (1.33)	11.02 (1.09)	18.92 (1.18)	27.59 (1.33)	5.84 (1.00)	15.92 (1.16)	23.59 (1.30)
	0.10	15.26 (1.12)	22.14 (1.23)	29.76 (1.36)	13.24 (1.10)	21.07 (1.23)	28.62 (1.36)	8.63 (1.08)	17.79 (1.21)	25.04 (1.35)

Note: The landmark time t′ for prediction is in bold. All the values are multiplied by 100.

TABLE 2.

Comparison of Brier score (BS) based on time-varying JHCM (TVJHCM), joint hierarchical copula modeling (JHCM), and JFM when the parameter θ_μ = 2, ρ = 0.5.

		Censoring rate = 0.6			Censoring rate = 0.5			Censoring rate = 0.4
n	t	TVJHCM (SD)	JHCM (SD)	JFM (SD)	TVJHCM (SD)	JHCM (SD)	JFM (SD)	TVJHCM (SD)	JHCM (SD)	JFM (SD)
100	0.03	3.92 (2.31)	8.44 (2.33)	17.72 (2.40)	3.69 (2.23)	8.10 (2.29)	16.99 (2.39)	2.19 (2.22)	6.76 (2.22)	13.44 (2.28)
	0.04	7.85 (2.31)	12.05 (2.34)	19.94 (2.42)	7.50 (2.24)	11.81 (2.30)	19.23 (2.41)	5.90 (2.23)	10.00 (2.23)	15.48 (2.30)
	0.05	10.66 (2.32)	15.70 (2.35)	22.03 (2.44)	10.25 (2.25)	15.26 (2.31)	21.09 (2.42)	8.94 (2.25)	12.93 (2.30)	17.12 (2.41)
	0.06	13.27 (2.33)	18.83 (2.36)	23.62 (2.45)	12.58 (2.26)	18.31 (2.32)	22.62 (2.43)	11.80 (2.26)	15.73 (2.31)	18.56 (2.42)
	0.07	15.29 (2.33)	21.62 (2.38)	25.12 (2.47)	14.37 (2.26)	20.88 (2.34)	23.94 (2.45)	13.88 (2.26)	18.35 (2.33)	19.80 (2.45)
	0.08	16.84 (2.35)	23.82 (2.39)	26.20 (2.48)	16.12 (2.26)	23.10 (2.35)	25.04 (2.46)	15.73 (2.26)	20.33 (2.33)	20.90 (2.46)
	0.09	18.12 (2.35)	26.07 (2.39)	27.26 (2.48)	17.35 (2.26)	25.12 (2.35)	25.98 (2.46)	17.43 (2.26)	22.31 (2.33)	22.02 (2.46)
	0.10	19.04 (2.36)	27.83 (2.41)	28.17 (2.49)	18.37 (2.30)	27.05 (2.36)	26.94 (2.49)	18.73 (2.29)	24.21 (2.34)	23.01 (2.48)
400	0.03	3.91 (1.11)	8.28 (1.20)	17.22 (1.32)	3.26 (1.09)	7.82 (1.18)	15.87 (1.30)	1.60 (1.06)	6.32 (1.16)	11.33 (1.28)
	0.04	7.79 (1.12)	11.95 (1.21)	19.65 (1.33)	7.18 (1.09)	11.32 (1.18)	18.09 (1.30)	5.63 (1.06)	9.41 (1.16)	13.13 (1.28)
	0.05	10.93 (1.14)	15.36 (1.23)	21.59 (1.37)	10.33 (1.10)	14.70 (1.20)	19.89 (1.33)	8.96 (1.07)	12.28 (1.19)	14.66 (1.30)
	0.06	13.19 (1.14)	18.44 (1.24)	23.12 (1.39)	12.74 (1.11)	17.68 (1.23)	21.41 (1.36)	12.03 (1.10)	14.95 (1.20)	16.05 (1.39)
	0.07	15.05 (1.15)	21.24 (1.26)	24.54 (1.40)	14.78 (1.11)	20.43 (1.25)	22.77 (1.38)	14.58 (1.10)	17.64 (1.23)	17.23 (1.40)
	0.08	16.59 (1.16)	23.67 (1.28)	25.76 (1.43)	16.42 (1.12)	22.68 (1.27)	23.90 (1.39)	16.75 (1.11)	19.85 (1.24)	18.38 (1.41)
	0.09	17.78 (1.16)	25.86 (1.29)	26.72 (1.45)	17.75 (1.14)	24.79 (1.29)	24.82 (1.40)	18.72 (1.12)	22.02 (1.26)	19.45 (1.44)
	0.10	18.71 (1.16)	27.80 (1.29)	27.59 (1.45)	19.06 (1.14)	26.70 (1.29)	25.75 (1.40)	20.60 (1.12)	24.01 (1.26)	20.55 (1.44)
100	0.06	8.64 (2.31)	15.66 (2.33)	25.01 (2.39)	8.00 (2.29)	15.36 (2.31)	23.92 (2.37)	4.74 (2.26)	12.57 (2.29)	20.04 (2.35)
	0.07	11.76 (2.32)	18.51 (2.34)	26.70 (2.40)	10.75 (2.30)	17.95 (2.32)	25.43 (2.38)	7.65 (2.25)	15.16 (2.27)	21.62 (2.34)
	0.08	14.17 (2.33)	20.87 (2.35)	27.92 (2.41)	13.31 (2.32)	20.31 (2.33)	26.66 (2.39)	10.20 (2.26)	17.18 (2.29)	23.02 (2.35)
	0.09	16.16 (2.33)	23.47 (2.35)	29.09 (2.42)	15.19 (2.32)	22.65 (2.33)	27.72 (2.40)	12.53 (2.26)	19.27 (2.29)	24.37 (2.38)
	0.10	17.69 (2.33)	25.50 (2.35)	30.07 (2.45)	16.76 (2.32)	24.81 (2.33)	28.80 (2.43)	14.35 (2.26)	21.24 (2.29)	25.58 (2.40)
400	0.06	8.68 (1.10)	15.22 (1.24)	24.36 (1.37)	7.27 (1.10)	14.49 (1.24)	22.95 (1.36)	3.53 (1.07)	11.66 (1.20)	17.60 (1.35)
	0.07	11.57 (1.11)	18.03 (1.25)	25.94 (1.39)	10.26 (1.11)	17.20 (1.25)	24.52 (1.38)	6.86 (1.08)	14.18 (1.21)	19.21 (1.38)
	0.08	13.96 (1.11)	20.62 (1.26)	27.25 (1.40)	12.66 (1.12)	19.50 (1.26)	25.79 (1.39)	9.70 (1.09)	16.27 (1.22)	20.75 (1.40)
	0.09	15.83 (1.12)	23.02 (1.28)	28.30 (1.44)	14.64 (1.13)	21.85 (1.27)	26.80 (1.41)	12.26 (1.10)	18.41 (1.23)	22.17 (1.40)
	0.10	17.36 (1.12)	25.21 (1.28)	29.23 (1.44)	16.54 (1.13)	23.96 (1.27)	27.77 (1.41)	14.68 (1.10)	20.38 (1.23)	23.53 (1.40)
100	0.09	13.12 (2.19)	21.20 (2.20)	29.96 (2.34)	12.19 (1.96)	20.34 (2.18)	28.59 (2.30)	6.97 (1.94)	16.84 (2.16)	25.52 (2.29)
	0.10	15.38 (2.19)	23.32 (2.20)	30.92 (2.35)	14.43 (1.96)	22.57 (2.18)	29.68 (2.33)	9.35 (1.94)	18.84 (2.16)	26.76 (2.30)
400	0.09	12.95 (1.07)	20.48 (1.19)	29.03 (1.30)	10.84 (1.06)	19.25 (1.16)	27.67 (1.29)	5.20 (1.05)	15.82 (1.16)	23.49 (1.28)
	0.10	15.16 (1.07)	22.68 (1.19)	29.95 (1.32)	13.41 (1.06)	21.35 (1.16)	28.62 (1.30)	8.19 (1.05)	17.68 (1.16)	24.88 (1.29)

Note: The landmark time t′ for prediction is in bold. All the values are multiplied by 100.

Regarding the discrimination ability quantified by the ROC-AUC (refer to Tables 3 and 4), we also observe the superiority of TVJHCM over JHCM and JFM. In particular, TVJHCM achieves more significant advantages over JHCM and JFM when the sample size increases (from 100 to 400), the censoring rate is relatively low (ie, 0.4), or there exists a stronger dependency with higher (temporal) variability between two event processes. For example, when $\rho$ is lower (ie, 0.1), a larger difference in ROC-AUCs between TVJHCM and JHCM can be observed, particularly as the sample size increases.

TABLE 3.

Comparison of receiving operating characteristic curve and the area under the curve (ROC-AUC) (%) based on time-varying JHCM (TVJHCM), joint hierarchical copula modeling (JHCM), and JFM when θ_μ = 1, ρ = 0.1.

		Censoring rate = 0.6			Censoring rate = 0.5			Censoring rate = 0.4
n	t	TVJHCM (SD)	JHCM (SD)	JFM (SD)	TVJHCM (SD)	JHCM (SD)	JFM (SD)	TVJHCM (SD)	JHCM (SD)	JFM (SD)
100	0.03	76.92 (3.21)	76.92 (3.11)	63.25 (4.33)	76.92 (3.11)	76.88 (3.01)	63.53 (4.21)	77.18 (1.99)	77.29 (2.67)	64.54 (4.00)
	0.04	77.21 (3.11)	77.21 (3.01)	64.29 (4.01)	77.28 (3.22)	77.25 (3.00)	63.45 (4.00)	77.58 (2.00)	77.71 (2.55)	64.39 (3.98)
	0.05	77.43 (3.09)	77.43 (3.00)	64.27 (4.00)	77.75 (2.98)	77.71 (2.97)	64.18 (4.01)	77.87 (1.88)	78.08 (2.54)	64.21 (3.88)
	0.06	77.60 (2.86)	77.60 (3.00)	64.21 (4.01)	77.90 (3.00)	77.86 (3.00)	64.04 (4.04)	78.07 (1.88)	78.22 (2.53)	64.02 (4.04)
	0.07	77.73 (3.33)	77.74 (3.22)	64.13 (4.32)	78.00 (3.09)	77.97 (3.05)	64.90 (4.44)	78.21 (1.88)	78.36 (2.89)	63.85 (3.65)
	0.08	77.84 (2.99)	77.84 (3.34)	64.05 (4.04)	78.00 (3.02)	77.97 (3.00)	64.90 (4.02)	78.31 (1.88)	78.46 (2.57)	63.68 (3.86)
	0.09	77.92 (2.66)	77.93 (2.88)	64.95 (3.67)	78.08 (2.80)	78.05 (2.99)	64.78 (3.76)	78.37 (2.00)	78.53 (2.56)	63.53 (3.98)
	0.10	77.99 (3.11)	77.99 (2.99)	64.84 (3.87)	78.14 (2.87)	78.10 (2.90)	64.64 (3.99)	78.41 (2.44)	78.57 (2.86)	63.41 (3.66)
400	0.03	78.92 (2.04)	74.67 (2.36)	65.67 (2.44)	77.97 (2.00)	77.93 (2.31)	64.92 (2.24)	79.82 (1.33)	74.99 (2.21)	65.67 (2.14)
	0.04	79.12 (2.10)	75.13 (2.38)	65.78 (2.49)	78.29 (2.02)	78.26 (2.38)	64.78 (2.26)	80.00 (1.37)	75.57 (2.33)	65.47 (2.20)
	0.05	79.26 (2.13)	75.49 (2.41)	65.84 (2.49)	78.51 (2.05)	78.48 (2.39)	64.65 (2.29)	80.12 (1.39)	76.00 (2.35)	65.38 (2.24)
	0.06	79.36 (2.15)	75.76 (2.43)	65.82 (2.50)	78.64 (2.06)	78.62 (2.40)	64.53 (2.31)	80.19 (1.41)	76.32 (2.37)	65.31 (2.28)
	0.07	79.44 (2.21)	75.98 (2.47)	65.91 (2.49)	78.73 (2.09)	78.71 (2.43)	64.42 (2.35)	80.23 (1.45)	76.56 (2.39)	65.18 (2.30)
	0.08	79.48 (2.25)	76.14 (2.50)	65.20 (2.43)	78.78 (2.11)	78.76 (2.46)	64.37 (2.45)	80.23 (1.49)	76.74 (2.41)	65.06 (2.36)
	0.09	79.51 (2.28)	76.27 (2.50)	65.20 (2.45)	78.80 (2.18)	78.78 (2.49)	64.30 (2.49)	80.22 (1.50)	76.88 (2.45)	65.01 (2.49)
	0.10	79.53 (2.30)	76.38 (2.50)	65.81 (2.50)	78.81 (2.21)	78.79 (2.49)	64.23 (2.49)	80.20 (1.55)	76.98 (2.48)	64.93 (2.49)
100	0.06	77.31 (2.66)	77.34 (2.88)	72.06 (3.82)	77.58 (2.86)	77.51 (2.78)	73.02 (3.91)	77.86 (1.58)	77.88 (2.53)	73.83 (3.84)
	0.07	77.46 (2.68)	77.49 (2.90)	71.97 (3.87)	77.74 (2.88)	77.67 (2.80)	72.86 (3.99)	78.03 (1.61)	78.05 (2.57)	73.68 (3.89)
	0.08	77.58 (2.69)	77.61 (2.91)	71.88 (3.88)	77.87 (2.89)	77.80 (2.83)	72.74 (3.99)	78.14 (1.63)	78.18 (2.59)	73.55 (3.90)
	0.09	77.68 (2.69)	77.71 (2.94)	71.81 (3.90)	77.96 (2.93)	77.89 (2.86)	72.66 (3.99)	78.24 (1.70)	78.28 (2.61)	73.45 (3.90)
	0.10	77.76 (2.71)	77.78 (2.96)	71.75 (3.93)	78.04 (2.93)	77.97 (2.88)	72.59 (3.99)	78.30 (1.79)	78.35 (2.61)	73.40 (3.92)
400	0.06	79.27 (1.49)	75.08 (1.50)	63.81 (1.98)	77.73 (1.44)	77.65 (1.50)	64.74 (2.95)	79.86 (0.98)	73.95 (1.50)	65.42 (1.88)
	0.07	79.35 (1.50)	75.32 (1.51)	81.84 (1.99)	64.88 (1.45)	77.80 (1.51)	74.66 (2.96)	80.01 (0.99)	74.34 (1.51)	65.40 (1.89)
	0.08	79.41 (1.53)	75.52 (1.53)	81.86 (2.00)	64.99 (1.46)	77.92 (1.52)	74.59 (2.97)	80.12 (1.00)	74.65 (1.52)	65.34 (1.93)
	0.09	79.45 (1.53)	75.68 (1.54)	64.86 (2.03)	75.07 (1.47)	78.00 (1.53)	74.53 (2.99)	80.13 (1.02)	74.90 (1.54)	65.31 (1.94)
	0.10	79.48 (1.53)	75.81 (1.54)	64.85 (2.03)	75.13 (1.47)	78.06 (1.53)	74.47 (2.99)	80.14 (1.02)	75.10 (1.54)	65.24 (1.94)
100	0.09	77.62 (2.66)	77.57 (2.92)	71.76 (3.89)	77.74 (2.92)	77.69 (2.84)	72.67 (3.95)	77.96 (1.68)	77.77 (2.58)	63.44 (3.88)
	0.10	77.71 (2.68)	77.85 (2.94)	71.69 (3.90)	77.82 (2.90)	77.77 (2.86)	72.61 (3.96)	78.03 (1.77)	77.86 (2.59)	63.35 (3.90)
400	0.09	79.41 (1.50)	75.05 (1.52)	71.83 (2.00)	77.94 (1.44)	77.88 (1.50)	74.62 (2.94)	79.36 (1.00)	73.72 (1.53)	65.38 (1.90)
	0.10	79.44 (1.50)	75.21 (1.52)	71.83 (2.00)	78.01 (1.45)	77.94 (1.50)	74.57 (2.96)	79.39 (1.00)	73.98 (1.53)	65.32 (1.90)

Note: The landmark time t′ for prediction is in bold.

TABLE 4.

Comparison of receiving operating characteristic curve and the area under the curve (ROC-AUC) (%) based on time-varying JHCM (TVJHCM), joint hierarchical copula modeling (JHCM), and JFM when θ_μ = 2, ρ = 0.5.

		Censoring rate = 0.6			Censoring rate = 0.5			Censoring rate = 0.4
n	t	TVJHCM (SD)	JHCM (SD)	JFM (SD)	TVJHCM (SD)	JHCM (SD)	JFM (SD)	TVJHCM (SD)	JHCM (SD)	JFM (SD)
100	0.03	76.67 (3.18)	76.36 (3.08)	64.11 (4.31)	77.94 (3.08)	78.30 (2.97)	65.37 (4.19)	77.94 (1.96)	78.30 (2.64)	65.37 (3.97)
	0.04	78.92 (2.99)	76.60 (2.99)	64.07 (3.98)	78.14 (3.20)	78.46 (2.98)	65.32 (3.98)	78.14 (1.98)	78.46 (2.53)	65.32 (3.88)
	0.05	77.11 (3.06)	76.77 (2.97)	65.95 (3.97)	78.27 (2.96)	78.56 (2.95)	65.24 (3.98)	78.27 (1.86)	78.56 (2.52)	65.24 (3.88)
	0.06	77.25 (2.84)	76.90 (2.98)	65.95 (3.98)	78.35 (2.98)	78.61 (2.99)	65.16 (4.01)	78.35 (1.86)	78.61 (2.51)	65.16 (4.01)
	0.07	77.34 (3.30)	76.99 (3.20)	65.89 (4.30)	78.39 (3.05)	78.63 (3.03)	65.09 (4.42)	78.39 (1.86)	78.63 (2.87)	65.09 (3.63)
	0.08	77.41 (2.97)	77.05 (3.32)	65.64 (4.02)	78.41 (3.00)	78.63 (2.98)	65.02 (4.00)	78.41 (1.86)	78.63 (2.55)	65.02 (3.84)
	0.09	77.46 (2.97)	77.09 (2.87)	65.79 (3.65)	78.41 (2.78)	78.62 (2.97)	65.96 (3.74)	78.41 (1.98)	78.62 (2.54)	65.96 (3.96)
	0.10	77.50 (3.08)	77.11 (2.97)	65.75 (3.85)	78.40 (2.85)	78.59 (2.87)	65.91 (3.97)	78.40 (2.42)	78.59 (2.84)	65.91 (3.64)
400	0.03	76.54 (2.01)	76.50 (2.34)	65.03 (2.42)	77.18 (1.97)	77.24 (2.29)	65.85 (2.22)	77.64 (1.30)	77.67 (2.19)	66.67 (2.12)
	0.04	76.83 (2.07)	76.78 (2.36)	65.09 (2.47)	77.50 (1.99)	77.57 (2.36)	65.75 (2.24)	77.99 (1.35)	78.01 (2.31)	66.49 (2.18)
	0.05	77.06 (2.10)	77.00 (2.39)	65.10 (2.47)	77.74 (2.03)	77.81 (2.37)	65.61 (2.26)	78.22 (1.37)	78.25 (2.33)	66.29 (2.21)
	0.06	77.25 (2.12)	77.18 (2.40)	65.06 (2.47)	77.91 (2.03)	77.81 (2.37)	65.61 (2.28)	78.39 (1.38)	78.41 (2.34)	66.10 (2.25)
	0.07	77.41 (2.18)	77.33 (2.44)	65.10 (2.46)	78.04 (2.06)	78.11 (2.40)	65.32 (2.32)	78.50 (1.42)	78.52 (2.36)	66.91 (2.27)
	0.08	77.54 (2.22)	77.46 (2.48)	65.95 (2.42)	78.13 (2.08)	78.20 (2.43)	65.20 (2.45)	78.57 (1.46)	78.59 (2.38)	66.74 (2.36)
	0.09	77.85 (2.25)	77.57 (2.48)	65.86 (2.45)	78.20 (2.15)	78.26 (2.47)	65.05 (2.49)	78.61 (1.46)	78.63 (2.43)	66.58 (2.49)
	0.10	77.75 (2.27)	77.66 (2.48)	65.77 (2.50)	78.24 (2.19)	78.30 (2.49)	65.91 (2.49)	78.63 (1.50)	78.85 (2.48)	66.45 (2.49)
100	0.06	75.66 (2.64)	75.48 (2.86)	63.05 (3.82)	77.18 (2.84)	77.78 (2.78)	64.30 (3.91)	77.18 (1.50)	77.78 (2.52)	64.33 (3.84)
	0.07	75.79 (2.65)	75.60 (2.90)	63.01 (3.87)	77.25 (2.86)	77.82 (2.80)	64.25 (3.99)	77.25 (1.56)	77.82 (2.54)	64.35 (3.89)
	0.08	75.90 (2.66)	75.70 (2.91)	63.98 (3.88)	77.30 (2.87)	77.83 (2.83)	64.20 (3.99)	77.30 (1.60)	77.83 (2.56)	64.21 (3.90)
	0.09	75.99 (2.66)	75.77 (2.94)	63.95 (3.90)	77.33 (2.90)	77.83 (2.86)	64.16 (3.99)	77.33 (1.67)	77.83 (2.58)	64.26 (3.90)
	0.10	76.06 (2.68)	75.83 (2.96)	63.92 (3.93)	77.34 (2.90)	77.82 (2.88)	64.12 (3.99)	77.34 (1.76)	77.82 (2.58)	64.22 (3.92)
400	0.06	77.14 (1.47)	77.06 (1.48)	64.91 (1.96)	77.89 (1.42)	77.76 (1.48)	66.35 (2.93)	78.15 (0.96)	78.14 (1.48)	65.97 (1.86)
	0.07	77.30 (1.48)	77.21 (1.50)	64.83 (1.98)	77.83 (1.48)	77.90 (1.49)	64.18 (2.94)	78.28 (0.97)	78.27 (1.49)	65.79 (1.87)
	0.08	77.44 (1.51)	77.33 (1.51)	64.73 (1.98)	77.94 (1.44)	78.00 (1.50)	64.03 (2.95)	78.37 (0.98)	78.36 (1.50)	65.64 (1.91)
	0.09	77.55 (1.51)	77.44 (1.52)	64.63 (2.01)	78.02 (1.45)	78.08 (1.51)	64.93 (2.97)	78.44 (1.00)	78.43 (1.52)	65.50 (1.92)
	0.10	77.65 (1.51)	77.53 (1.52)	64.54 (2.00)	78.08 (1.45)	78.14 (1.52)	64.86 (2.97)	78.47 (1.00)	78.48 (1.52)	65.38 (1.92)
100	0.09	76.54 (2.64)	77.20 (2.90)	64.99 (3.86)	76.76 (2.90)	77.39 (2.82)	64.22 (3.93)	76.76 (1.66)	77.39 (2.56)	64.22 (3.86)
	0.10	76.59 (2.66)	77.22 (2.92)	64.97 (3.88)	76.77 (2.90)	77.37 (2.90)	64.19 (3.98)	76.77 (1.78)	77.37 (2.61)	64.19 (3.92)
400	0.09	77.34 (1.46)	77.27 (1.48)	64.52 (1.98)	77.89 (1.40)	77.93 (1.46)	64.91 (2.90)	78.22 (0.96)	78.18 (1.49)	65.45 (1.86)
	0.10	77.45 (1.46)	77.37 (1.48)	64.45 (1.98)	77.96 (1.40)	78.00 (1.46)	64.83 (2.90)	78.28 (0.96)	78.24 (1.49)	65.39 (1.86)

Note: The landmark time t′ for prediction is in bold.

We acknowledge that the patterns identified above may be slightly violated in certain cases, potentially due to small sample sizes or numerical issues in finite samples. However, these mild deviations do not affect our overall conclusion. Note that (paired) $t$-tests are used to detect statistically significant differences between the models, and we realize that: (1) there are significant differences in predictive accuracy between TVJHCM and JFM, as well as between JHCM and JFM in all scenarios; and (2) significant differences in predictive accuracy exist between TVJHCM and JHCM in all scenarios except for cases with a landmark time of $t^{\prime }=0.03$, $n=100$, ${\theta }_u=1$, $\rho =0.1$, and censoring rate = 0.6. Furthermore, we conduct additional simulation studies to evaluate the robustness of TVJHCM by considering scenarios with a time-invariant dependence between two event processes and those with mis-specification of copula functions (see the Data S1). We find that TVJHCM performs comparably to JHCM when the dependence of event processes is constant, and the performance of TVJHCM remains satisfactory, although the bias tends to be slightly larger when the copula is incorrect. As recommended by the reviewers, we perform additional empirical analyses to thoroughly evaluate and compare our models. First, we generate data using more general mechanisms that are not aligned with any of the three models, in order to ensure a more equitable comparison. Our findings indicate that the TVJHCM model performs well and exhibits comparable predictive performance to the other models. Second, we conduct a dedicated calibration assessment to investigate the possibility of overfitting in finite sample sizes. The results, which are presented in the Supplementary Materials due to space constraints, suggest no evidence of overfitting, based on both internal and external validation.

Overall, our results suggest that TVJHCM has the best performance with robustness to the mis-specification of copula and comparability to JHCM under the case with constant correlation between terminal and recurrent events. Of note, we develop the R package of BJHCM to adopt our proposed methods, which is available at https://github.com/Maggie8888/BJHCM.

4 |. DATA APPLICATION

To illustrate the method, we will analyze data from two large prospective epidemiological studies: the CHS and the Atherosclerosis Risk in Communities (ARIC) study. We will assess the predictive accuracy of our method on all-cause mortality by conducting both internal and external validation.

4.1 |. The CHS and ARIC studies

The CHS study is an observational study funded by the National Heart, Lung, and Blood Institute (NHLBI), including a total of 5888 participants recruited from four U.S. communities (Forsyth County, North Carolina; Sacramento County, California; Washington County, Maryland; Pittsburgh, Pennsylvania) since 1989. The study aims to improve our understanding of cardiovascular risk factors in older adults.³² Eligible participants were noninstitutionalized individuals aged 65 years or older at the time of examination, expected to remain in the sampling area for the next 3 years, and able to provide informed consent. Annual clinical visits and phone contact at six-month intervals were used to collect information on hospitalizations and potential cardiovascular events. More details of the study design and rationale can be found at https://chs-nhlbi.org/CHSDesc. We focus on the events of MI, stroke, and mortality as primary outcomes, and explore several baseline risk factors, including gender (GEN; 1 = male, 0 = female), race (RACE; 1 = white, 0 = non-white), systolic blood pressure (SBP), baseline MI (MIB; 1 = yes, 0 = no), and stroke status (STB; 1 = yes, 0 = no). After excluding participants with missing data, our sample size for analysis is 5781.¹⁷ Patients were followed up to 18 years, with a death rate of 83.65% by the year 2006. The numbers of both MI and stroke recurrent events ranged from 0 to 5 across subjects.

The ARIC study for external validation, also funded by the NHLBI, is a cohort study that began recruiting black and white adults aged 45to 64 years from four U.S. communities (Forsyth County, North Carolina; Jackson, Mississippi; Minneapolis, Minnesota; Washington County, Maryland) in 1987. The study was designed to investigate the etiology and natural history of atherosclerosis, as well as variations in cardiovascular risk factors and disease.³³ Participants were followed up through prescheduled clinic examinations and semi-annual telephone contacts to provide risk factor and health status information. Our external validation uses the same set of risk factors as the CHS study, and after excluding missing data, we have a total of 8421 subjects in our analytic sample. All patients have been followed up for up to 27 years in the study, with a death rate of 70.76% by the year 2015.³³ More detailed information on this study can be found at https://biolincc.nhlbi.nih.gov/studies/aric-non/.

4.2 |. Dynamic prediction with internal and external validation

The probability of all-cause mortality is calculated based on historical event information collected up to a landmark time $t^{\prime }$. The maximum prediction horizon is set at 15 years to ensure a sufficient sample size for more reliable prediction. The accuracy of predicted survivals quantified using BS and ROC-AUC at a sequence of horizon times up to 15 years are recorded and compared.

We fit TVJHCM and JHCM using a Clayton copula and evaluate the predictive accuracy through internal and external cross-validation to prevent over fitting. Specifically, we use the CHS dataset for internal validation, where 1000 participants are randomly sampled in each cross-validation process, and further randomly split data into training and testing datasets with the 60% vs 40% ratio. This process is replicated 40 times for results summary, with each iteration using the training dataset for model fitting and the testing dataset for prediction. Regarding external validation, we estimate the model using the CHS study and then randomly sample500 subjects from the ARIC study as the testing dataset. We perform this process across 40 resampled datasets to summarize the results.

4.3 |. Results for data application

The results of model fitting for the CHS study are summarized in Table 5. Baseline MI or stroke, gender, and SBP are identified as having significant associations with the outcomes in all models, and TVJHCM can detect more racial disparities compared to the other two alternatives. More importantly, both joint models exhibit relatively high temporal correlation estimates for MI ($\widehat{\rho }=0.660$ [95% CI:0.602, 0.720]) and stroke ($\widehat{\rho }=0.995$ [95% CI:0.978, 1.000]), indicating less variability in the dependency across time between event processes, particularly for stroke and death. Figure 1 presents individual-level dynamic predictions based on TVJHCM and JHCM, using a randomly selected subject from the CHS study for illustration. This subject experienced 13 recurrent events of stroke during follow-up and was censored around 12 years. We observe an increasing risk of death over time, with a relatively higher risk of death obtained from JHCM. Furthermore, the increase in the risk of death tends to be slower for earlier landmark times (top in Figure 1). Additionally, as more recurrent events accumulate, the mortality risk increases (comparing bottom to top in Figure 1).

TABLE 5.

Parameter estimates for joint modeling of recurrent MI or stroke and death for the Cardiovascular Health Study.

		TVJHCM	JHCM	JFM
Joint modeling of MI and death
	Covariate	HR (2.5%, 97.5%)	HR (2.5%, 97.5%)	HR (2.5%, 97.5%)
Time to death	MI history yes	1.568(1.439, 1.717)	1.581(1.477, 1.710)	1.632(1.495, 1.782)
	Gender male	1.308(1.230, 1.384)	1.332(1.261, 1.394)	1.336(1.255, 1.422)
	Race white	1.052(0.995, 1.171)	0.991(0.961, 1.023)	1.008 (0.926,1.097)
	SBP	1.160(1.127, 1.195)	1.158(1.128, 1.189)	1.164(1.129, 1.200)
Time to MI	MI history yes	2.343(2.132, 2.654)	2.381(2.107, 2.795)	2.444(2.159, 2.767)
	Gender male	1.646(1.497, 1.769)	1.712(1.588, 1.874)	1.690(1.528, 1.872)
	Race white	1.580(1.351, 1.916)	1.128(1.033, 1.228)	1.145(0.991, 1.321)
	SBP	1.240(1.183, 1.295)	1.247(1.185, 1.314)	1.238(1.179, 1.301)
	${\sigma }_{\omega }^2$	0.004(0.003, 0.005)	0.009(0.007, 0.011)
	${\sigma }_{\epsilon }^2$	0.320(0.188, 0.433)
		EST(2.5%, 97.5%)	EST(2.5%, 97.5%)
Copula parameter	${\theta }_{\mu }$	0.265(0.240, 0.293)	0.302(0.271, 0.333)
AR(1) correlation	$\rho$	0.660(0.602, 0.720)
Joint modeling of Stroke and death
	Covariate	HR(2.5%, 97.5%)	HR(2.5%, 97.5%)	HR(2.5%, 97.5%)
Time to death	Stroke history yes	1.490(1.388, 1.607)	1.490(1.370, 1.613)	1.659(1.521, 1.809)
	Gender male	1.314(1.229, 1.407)	1.327(1.254, 1.387)	1.345(1.264, 1.430)
	Race white	1.156(1.053, 1.258)	1.012(0.961, 1.077)	1.009(0.928, 1.099)
	SBP	1.161(1.130, 1.189)	1.159(1.133, 1.188)	1.173(1.137, 1.210)
Time to Stroke	Stroke history yes	2.098(1.867, 2.352)	2.132(1.864, 2.426)	2.445(2.141, 2.794)
	Gender male	1.134(1.032, 1.242)	1.126(1.020, 1.236)	0.962(0.864, 1.072)
	Race white	1.334(1.111, 1.742)	1.053(0.963, 1.184)	1.030(0.894, 1.187)
	SBP	1.335(1.276, 1.404)	1.333(1.284, 1.396)	1.373(1.306, 1.444)
	${\sigma }_{\omega }^2$	0.002(0.001, 0.002)	0.010(0.008, 0.012)
	${\sigma }_{\epsilon }^2$	0.0001(0.0000, 0.0009)
		EST(2.5%, 97.5%)	EST(2.5%, 97.5%)
Copula parameter	${\theta }_{\mu }$	0.503(0.477, 0.523)	0.481(0.452, 0.510)
AR(1) correlation	$\rho$	0.995(0.978, 1.000)

Abbreviations: JHCM, joint hierarchical copula modeling; TVJHCM, time-varying joint hierarchical copula modeling.

FIGURE 1.

Example of individual-level prediction based on time-varying joint hierarchical copula modeling (TVJHCM), joint hierarchical copula modeling(JHCM), and JFM.

In Figure 2, we can observe the benefits of TVJHCM on death prediction for patients with MI or stroke, and notice that the BS is lower at earlier prediction times and then increases over time for a fixed landmark time. Table 6 presents the time-dependent ROC-AUC values for each prediction time $t$ considering internal and external validation strategies. Due to the relatively stable dependency across time between event processes, TVJHCM and JHCM exhibit comparable discrimination ability, but both perform better than JFM. This finding is consistent with the simulation studies presented in Table 4. Overall, both models demonstrate acceptable ROC-AUCs (ranging from 0.74 to 0.85) and small BS (less than 0.50), indicating good prediction performance in discrimination and calibration. It is worth noting that JHCM outperforms TVJHCM in terms of BS, especially for external validation using the ARIC study.

FIGURE 2.

The Brier scores (BS) for MI (top left) and stroke (top right) based on internal validation using the CHS study, and also for MI (bottom left) and stroke (bottom right) based on external validation using the ARIC studies. The black lines are results from JHCM, the blue lines are from JFM and the red lines are from TVJHCM.

TABLE 6.

Comparison of receiving operating characteristic curve and the area under the curve (ROC-AUC) (%) based on time-varying JHCM (TVJHCM), joint hierarchical copula modeling (JHCM), and JFM using the Cardiovascular Health Study (CHS) study for internal validation and using the Atherosclerosis Risk in Communities (ARIC) study as testing dataset for external validation.

	Internal validation						External validation
	Stroke			MI			Stroke			MI
t (year)	TVJHCM	JHCM	JFM	TVJHCM	JHCM	JFM	TVJHCM	JHCM	JFM	TVJHCM	JHCM	JFM
6	76.45	77.22	76.22	73.55	73.44	73.34	85.39	84.26	83.56	78.34	78.21	77.79
7	76.44	77.11	76.21	73.21	73.22	72.66	85.33	84.11	83.44	77.22	78.11	77.33
8	76.55	77.01	76.11	77.51	77.33	72.34	85.23	84.04	83.14	77.22	79.44	76.90
9	75.63	77.44	75.31	77.51	77.33	72.01	84.34	83.33	82.31	76.11	79.22	76.21
10	75.44	77.12	75.23	77.44	77.22	72.81	84.44	83.33	82.31	76.33	75.44	75.12
11	75.41	77.01	75.21	70.22	70.01	70.01	84.22	83.21	82.31	76.22	75.44	74.65
12	75.44	77.21	75.19	78.55	78.22	69.69	84.21	83.21	81.09	76.21	75.21	74.36
13	75.44	77.22	74.54	78.54	78.11	69.31	84.33	83.19	81.00	75.21	75.21	74.31
14	75.51	77.21	74.54	78.53	78.10	69.23	84.29	83.11	81.00	74.89	74.99	74.31
15	75.45	77.11	74.01	78.44	78.10	68.56	84.21	82.99	81.01	74.22	74.84	73.66
9	75.21	75.11	74.55	79.21	79.10	78.65	81.21	81.11	80.99	79.45	80.11	80.45
10	75.22	75.10	74.49	79.19	79.09	78.09	81.22	81.09	80.01	74.22	78.91	77.22
11	75.22	75.00	74.49	78.11	78.05	77.89	81.21	81.10	80.01	74.22	78.10	77.22
12	75.21	75.00	74.41	78.09	78.04	77.89	81.10	81.01	80.01	74.21	77.90	73.99
13	75.22	75.01	74.01	78.08	78.01	77.71	81.11	80.10	80.00	74.54	76.22	73.99
14	75.22	74.91	73.91	75.01	74.97	73.17	81.11	80.09	79.99	74.89	75.68	73.91
15	75.22	74.65	74.51	71.45	71.21	70.88	81.11	80.10	79.96	74.11	71.89	71.29
12	75.45	75.21	74.91	74.47	75.01	74.31	79.20	80.11	79.01	77.90	81.11	76.65
13	75.42	75.11	74.91	74.44	75.01	74.31	79.11	80.10	79.01	76.44	76.23	75.45
14	75.44	75.22	73.22	75.11	75.01	74.00	79.11	80.11	79.01	76.21	75.10	74.01
15	75.44	75.21	73.01	75.10	75.00	74.00	79.12	80.01	79.00	76.11	74.66	73.91

Although TVJHCM utilizes more information as $t$ increases, leading to more accurate predictions, a decrease in discrimination is expected, as observed in the simulation studies. This may be due to a selection process that makes the at-risk population more homogeneous as $t$ increases. Prediction errors are similar when comparing TVJHCM with JHCM, but TVJHCM has a smaller prediction error in internal validation using the CHS data only. However, we notice an increase in accuracy when we use the ARIC data as our external validation dataset (Figure 2). This indicates that considering the time-varying correlation between terminal and recurrent events in the prediction of death could result in more accurate predictions. We also assess the calibration performance of our models in the real data application. As shown in Figure 3, TVJHCM demonstrates satisfactory results with no indications of overfitting.

FIGURE 3.

Calibration plots of internal (left panel) and external (right panel) predictions for time-varying joint hierarchical copula modeling (TVJHCM) (top), joint hierarchical copula modeling (JHCM) (middle) and JFM (bottom) with Cardiovascular Health Study data as training and Atherosclerosis Risk in Communities data as testing datasets. The results are for prediction at 9 years.

5 |. DISCUSSION

The methodology presented in this article addresses recurrent events in the presence of a terminal event (eg, death), which are commonly encountered in practical applications. Our (time-varying) joint hierarchical copula model differs from traditional joint frailty models in that it relaxes the assumption of conditional independence, enabling the incorporation of (time-dependent) correlations between the processes of recurrent and terminal events due to unobserved time-varying covariates. In this study, we focus exclusively on evaluating the predictive ability of our proposed model using discrimination and calibration metrics based on ROC-AUC and BS as well as calibration plots in a dynamic manner. We employ the random sub-resampling (eg, bootstrap sampling) validation method, which we found to perform better than K-fold cross-validation under finite samples in our empirical studies. However, other cross-validation techniques, such as stratified K-fold cross-validation, are also worth considering for further validation.

From numerical studies through simulations and real data applications, TVJHCM has been shown to efficiently model recurrent gap times and terminal event times with enhanced prediction accuracy. Notably, extensive simulation studies on calibration assessment did not reveal any overfitting issues. Additionally, this model can be implemented with satisfactory computational efficiency. For instance, in our data application using the CHS dataset, the average computing time for one iteration of internal cross-validation for TVJHCM is approximately 2.21 minutes, and the average computing time for model fitting using a dataset with a sample size of 100 is around 1.30 seconds. Incorporating the subject-level time-varying copula parameter ${\theta }_{ij}$ improves overall model fitting and predictive performance. When additional measurements are available, the predictions can be updated with improved accuracy and efficiency. Further, it is beneficial to incorporate multiple-type recurrent events to enhance the prediction and meet clinical needs. Our proposed methods are capable and flexible enough to extend to such contexts, which is one of our ongoing research interests. Our ultimate goal is to improve earlier diagnosis for subjects at a high predicted risk of deterioration, allowing for timely intervention to delay the manifestation of MI or stroke in patients with a history of MI or stroke.

There are limitations to our approach, which we plan to address in future work. Firstly, we ignore the fact that the last recurrent gap time could be censored by the terminal event. While some imputation methods exist in the literature to handle missing data in scalar predictors, it is challenging to deal with missing data in our proposed time-varying copula model, both in terms of statistical theory and computation. In future work, we will extend our Bayesian TVJHCM to account for missing data in both scalar and functional covariates. Secondly, in this article, we use time-invariant baseline data as predictors. It would be of scientific interest to expand the proposed Bayesian TVJHCM to accommodate longitudinal data. However, this would significantly increase computational costs, particularly when making dynamic predictions at the individual level. Thirdly, the correct specification of the copula function could enhance parameter estimation and risk prediction, although our previous work has demonstrated that our models are relatively robust to copula mis-specification. How to select an appropriate copula function is still of importance, and the utilization of modified deviance information criteria (eg, Bayes information criterion) could be helpful to assess and compare the copulas.³⁴ Additionally, the Bayesian model averaging technique can be considered,³⁵ where the weight of the candidate model based on the data can be updated, and the covariate effect estimates could be a weighted average from different candidate copula models. The proposed TVJHCM can be adapted to various scenarios and research purposes. Furthermore, the Bayesian procedure we develop here has demonstrated good inferential properties in simulation studies and real data applications. However, as sample sizes increase, computation time may become a substantial concern. Thus, parallelizing MCMC and variation approximations may address the computational concerns while providing satisfactory parameter estimation, which warrants future research.

Supplementary Material

SUPPORTING INFORMATION

Additional supporting information can be found online in the Supporting Information section at the end of this article.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available upon request which should be directed to the National Heart, Lung, and Blood Institute (NHLBI) Biologic Specimen and Data Repository Information Coordinating Center (https://biolincc.nhlbi.nih.gov/home/).